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6.  Abstract 

This  report  documents  an  evaluation  of  Aircraft  Separation  Assurance  (ASA)  con¬ 
cepts  using  an  airline  flight  simulator.  The  primary  objective  of  the  experiment 
was  to  determine  the  cockpit  information  requirements  for  an  aircraft  collision 
avoidance  system.  Qualified  pilots  from  commercial  aviation  and  industry  flew 
typical  operational  scenarios  in  the  simulated  Los  Angeles  area.  During  the 
flight,  conflict  situations  with  other  aircraft  developed,  and  pilots  were 
asked  to  respond  to  these  situations  on  the  basis  of  information  presented  to 
them  by  one  of  three  experimental  collision  avoidance  displays.  Computer-collected 
data  on  pilot  response  to  collision  avoidance  commands  and  resultant  miss  distances 
were  correlated  with  data  from  questionnaires  filled  out  by  participating  pilots  to 
determine  the  cockpit  information  requirements.  The  subjective  comments  addressed 
the  areas  of  required  display  information  items,  use  of  color,  audible  alert,  sym¬ 
bols,  workload,  pilot  confidence  in  the  system,  and  pilot  display  preference. 
Altitude,  range,  relative  bearing,  and  other-aircraft  heading  have  been  identified 
as  the  most  important  and  most  essential  information  elements  in  the  resolution 
of  potential  conflicts.  The  report  provides  a  statistical  analysis  of  the  accu¬ 
mulated  data  and  includes  recommendations  for  the  development  and  operational 
implementation  of  the  ASA  program. 
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FOREWORD 


A  study  of  the  cockpit  information  requirements  for  an  airborne  col¬ 
lision  avoidance  system  has  been  conducted  for  the  Systems  Research  and 
Development  Service  of  the  Federal  Aviation  Administration  under  Contract 
DOT-FA78-4091 .  This  report  documents  the  results  of  this  study.  Further 
experimentation  and  operational  flight  tests  with  specific  Aircraft  Separa 
tion  Assurance  System  equipment  should  be  conducted  to  confirm  these 
findings.  The  results  of  this  study  are  presented  in  two  volumes. 

Volume  I  -  Study  Report 
Volume  II  -  Appendixes 
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SUMMARY 


1.  INTRODUCTION 

During  December  1978  and  January  1979,  74  pilots,  representing  12 
airlines,  government  agencies,  and  fleet  operators,  participated  in  a  cock¬ 
pit  simulator  evaluation  of  a  generic  collision  avoidance  system.  This 
evaluation  is  part  of  the  Federal  Aviation  Administration's  (FAA)  Aircraft 
Separation  Assurance  (ASA)  Program. 

The  major  objective  of  the  evaluation  was  to  determine  the  information 
requirements  for  an  airborne  collision  avoidance  system  and  to  record  and 
measure  the  responses  of  air  carrier  flight  crews  to  a  realistic  simulation 
of  potential  midair  collisions.  An  equally  important  objective  of  the  pro¬ 
gram  was  to  prove  that  a  test  bed  using  an  airline  cockpit  simulator  could 
be  developed  for  evaluation  of  ASA  concepts  and  systems.  The  successful 
completion  of  this  program  is  proof  of  the  achievement  of  this  basic 
objective. 

2.  DESCRIPTION  OF  THE  EXPERIMENT 

♦ 

A  simulation  test  bed  was  set  up  at  the  United  Airlines  Flight  Training 
Center  in  Denver,  Colorado,  using  a  Boeing  727  cockpit  simulator  that 
featured  a  computer-generated  night  visual  scene.  This  system  was  modified 
so  that  other  simulated  aircraft  would  appear  when  their  position  relative 
to  the  simulator  cockpit  was  within  the  forward  windshield  area.  Figure  S-l 
illustrates  the  test-bed  layout.  A  simulation  control  computer  was  inter¬ 
faced  to  the  simulator  to  provide  an  air  traffic  control  (ATC)  simulation 
environment.  Progress  was  monitored  and  the  simulation  was  controlled 
through  interactive  terminals.  Pilots  flew  typical  scenarios  in  the  simu¬ 
lated  Los  Angeles  area  and  were  exposed  tc  each  of  three  different  collision 
avoidance  displays  installed  in  the  cockpit  simulator.  The  selected  ASA 
display  devices  included  a  modified  Instantaneous  Vertical  Speed  Indicator 
(IVSI) ,  a  Light  Emitting  Diode  (LED)  Display,  and  a  Cathode  Ray  Tube  (CRT) 
Display.  The  IVSI  used  lighted  indicators  to  display  only  ASA  commands. 

The  LED  display  presented  ASA  commands  and  traffic  advisories  in  alphanu¬ 
meric  form.  The  CRT  display  provided  a  graphical  presentation  of  ASA  com¬ 
mands  and  traffic  advisories. 
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Computer 


Volunteer  pilots,  solicited  from  the  aviation  community,  were 
provided  with  an  advance  briefing  packet  that  described  the  experiment, 
the  test  bed  setup,  the  test  scenarios,  and  administrative  information. 

Upon  arrival  at  the  test  site,  each  crew  received  a  formal  briefing  that 
reviewed  the  advance  material  and  included  a  slide  presentation  and  video¬ 
tape  on  the  collision  avoidance  system's  displays. 

The  test  session  consisted  of  three  scenarios  that  represented  typical 
night  flights  departing  Los  Angeles  International  Airport  for  a  nearby 
destination.  The  crew  received  a  clearance  from  the  air  traffic  controller 
and  made  a  standard  departure .  Radio  contact  was  maintained  throughout 
the  flight,  which  proceeded  outbound  for  approximately  20  minutes,  then 
was  cleared  back  to  Los  Angeles  airport  along  a  route  normally  used  for 
inbound  traffic.  The  turnaround  was  explained  in  the  preflight  briefing 
and  was  therefore  anticipated  by  the  crew.  Each  flight  consisted  of  all 
phases  encountered  during  an  operational  flight:  takeoff,  departure,  en 
route,  approach,  and  landing. 

During  the  flight,  the  crew  was  confronted  with  six  preplanned  collision 
situations  that  generally  resulted  in  a  commanded  action.  This  action  was 
shown  with  supporting  traffic  information  on  the  appropriate  collision  avoid¬ 
ance  display.  Pilots  were  instructed  (in  the  pre-flight  briefing)  to  respond 
to  the  display  by  using  standard  aircraft  maneuvers.  The  simulator  was  the 
only  aircraft  in  the  scenario  that  was  equipped  with  an  ASA  system. 

Data  were  collected  on  simulator  position,  velocity,  and  altitude; 
collision  avoidance  display  status;  and  pilot  response.  In  addition,  sub¬ 
jective  pilot  comments  were  collected  in  a  debriefing  session  immediately 
following  each  test  session.  The  debriefing  included  completing  a  lengthy 
questionnaire. 


3.  DATA  ANALYSIS 

The  data  collected  were  analyzed  to  determine  the  acceptability  of  the 
Aircraft  Separation  Assurance  concepts  tested  during  the  simulation  session. 
The  subjective  data  developed  from  the  questionnaires  and  comments  and  the 
performance  data  derived  from  the  recorded  quantitative  data  were  evaluated 
to  develop  the  results  and  conclusions  of  this  study.  Qualitative  data 
were  used  primarily  to  develop  pilot  reaction  to  questions  of  display  read¬ 
ability,  comprehension,  color,  format,  information  elements,  information 
usefulness,  and  impact  of  the  ASA  function  on  flight  and  control  procedures. 
Quantitative  data  provided  measurements  of  pilot  response  to  ASA  commands 
and  resulting  miss  distances. 


4.  CONCLUSIONS 

The  test  subjects  provided  clear  statements  of  their  opinions  regarding 
the  ASA  concepts  tested.  The  significant  results  are  summarized  as  follows: 

•  Information  Elements.  From  a  list  of  11  potential  collision  avoid¬ 
ance  information  items,  the  test  subjects  rated  4  items  as  most 
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essential.  None  of  the  remaining  7  items  were  selected  as 
essential  by  more  than  21  percent  of  the  test  subjects.  The  items 
listed  below  the  dashed  dividing  line  in  Table  S-l  can  be  cate¬ 
gorized  as  insignificant  in  the  view  of  the  respondents. 


Table  S-l.  RANKING  OP  INFORMATION  ITEMS 

Percent 

Rank 

Information  Item 

Responding 

"Essential" 

nu 

Altitude  of  other  aircraft 

85 

Range  of  other  aircraft 

81 

Relative  bearing 

59 

BBH 

Heading  of  other  aircraft 

51 

5 

Horizontal  closure  rate 

21 

6 

Vertical  closure  rate 

17 

7 

Vertical  speed  of  other  aircraft 

20 

8 

Projected  miss  distance 

14 

mm 

Closure  angle 

7 

Other  aircraft  type 

5 

Ka 

Other  aircraft  identity 

3 

•  Altitude  Representation.  The  overwhelming  majority  (89  percent) 
favored  a  mean  sea  level  (MSL)  or  absolute  representation  of 
altitude  versus  a  relative  representation. 

•  Traffic  Advisories.  More  than  79  percent  of  the  pilots  considered 
traffic  advisories  an  essential  part  of  an  ASA  display.  The  test 
subjects  found  that  traffic  advisories  were  useful  in  preventing  a 
potential  collision  from  developing  or  in  resolving  a  potential 
collision.  When  the  IVSI  display  was  used,  more  collision  avoid¬ 
ance  commands  resulted  because  of  the  lack  of  traffic  advisory 
information.  The  traffic  advisories  were  used  to  locate  and 
identify  other  traffic.  The  test  subjects  preferred  ASA-generated 
traffic  advisories  to  those  issued  by  air  traffic  controllers 
because  of  the  continuous  updating  capability  of  ASA.  This  was 
beneficial  in  both  acquiring  the  correct  aircraft  and  keeping 
traffic  in  sight.  However,  pilots  believed  that  a  display  of 
traffic  advisories  should  be  limited  to  the  two  or  three  most 
important . 

•  Display  Preference.  Of  the  three  types  of  displays  used  in  the 
experiment,  the  CRT  display  was  most  popular  (49  percent  of  the 
subject  pilots  rated  it  number  one) .  The  other  two  displays,  LED 
and  IVSI,  received  an  even  share  of  the  remaining  votes.  A  signif¬ 
icant  point,  however,  is  that  more  than  90  percent  of  the  test 
subjects  found  their  second  choice  acceptable  and  73  percent  found 
their  third  choice  acceptable.  Pilots  feel  strongly  that  something 
is  needed  now. 
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Alternative  Display  Techniques.  When  pilots  were  asked  to  suggest 
alternative  instruments  that  could  be  modified  to  display  collision 
avoidance  information,  the  most  commonly  recommended  devices  were 
the  horizontal  situation  indicator  (HSI/EHSI) ,  the  flight  director, 
and  the  weather  radar. 

Use  of  Color.  The  use  of  color  aids  in  the  interpretation  of  the 
traffic  advisories  and  commands.  The  use  of  red  for  positive 
maneuver  commands  is  acceptable. 

Audible  Alert.  There  is  almost  complete  agreement  that  an  audible 
alert  is  required  when  the  ASA  system  detects  a  potential  conflict. 

There  is  a  universal  belief  that  there  are  already  too  many  audible 
alarms  in  airline  cockpits;  however,  the  general  feeling  of  the  test 
subjects  is  that  a  life-threatening  situation  justifies  an  additional 
alarm. 

Use  of  Symbols.  The  pilots  preferred  symbols  over  text  for  positive 
commands  if  only  one  method  of  presentation  could  be  provided. 

However,  they  had  a  clear  preference  for  a  combination  of  both. 

Workload.  The  test  subjects  believed  that  the  introduction  of  ASA 
would  result  in  an  acceptable  increase  in  the  cockpit  workload  for 
the  nonflying  pilot.  This  increase  would  be  greatest  for  the  more 
complex  displays  that  require  close  monitoring.  The  pilots  also 
believed  that  the  communications  workload  would  increase  because  of 
a  greater  number  of  inquiries  to  ATC  from  flight  crews  to  coordinate 
maneuvers  resulting  from  conflicts  and  ASA-displayed  traffic  advisories. 

Horizontal  Versus  Vertical  Maneuvers.  There  was  no  strong  preference 
for  either  horizontal  or  vertical  avoidance  maneuvers  during  the 
climb,  cruise,  and  descent  phases  of  flight;  however,  the  pilots  pre¬ 
ferred  horizontal  maneuvers  during  takeoff,  approach,  and  landing. 

The  passenger  comfort  factor  was  most  often  mentioned  by  those  who 
preferred  horizontal  maneuvers. 

Excessive  Maneuver  Rates.  The  test  subjects  generally  believed 
that  they  did  not  need  to  use  unusual  maneuver  rates  to  resolve  a 
conflict.  Simulator  position  measurements  confirmed  that  the 
majority  of  the  pilots  used  standard  turn  and  pitch  rates  during 
resolution  maneuvers.  Most  banks  were  less  than  30°,  and  maximum 
pitch  rates  were  less  than  2.5°  per  second. 

Pilot  Confidence.  The  pilots  expressed  confidence  in  the  capability 
of  the  ASA  system  to  resolve  potential  conflicts.  They  felt  more 
confident  in  underflying  and  overflying  other  aircraft  with  1,000 
feet  separation;  however,  the  majority  believed  that  current  IFR 
separation  criteria  should  be  retained. 

Pilot  Response  Time.  The  pilots  considered  their  response  time  to 
be  independent  of  the  type  of  display  in  use;  however,  the  IVSI 
produced  the  shortest  average  response  time  to  commands  (2.996 
seconds) ,  and  the  CRT  displayed  commands  resulted  in  the  longest 
average  response  time  (3.468  seconds).  The  average  response  time 
for  the  LED  display  (3.123  seconds)  was  quite  close  to  that  of  the 
IVSI.  The  difference  was  due  to  both  the  location  of  the  CRT  dis¬ 
play  and  the  more  complex  symbology  used  on  that  display.  Regardless 


of  the  display  used,  the  response  to  the  first  command  in  a 
multicommand  sequence  took  an  average  of  1/2  second  longer 
than  the  response  to  subsequent  commands  in  the  sequence.  There 
was  a  statistically  significant  difference  in  the  response  time  to 
commands  for  vertical  maneuvers  (climb  and  descend)  as  compared 
to  commands  for  horizontal  maneuvers.  Response  to  commands  was 
somewhat  faster  for  vertical  versus  horizontal  maneuvers,  but  the 
difference  was  only  1  second,  which  does  not  have  a  major  impact 
on  the  system  design. 

•  Miss  Distance.  Seven  of  the  516  conflict  situations  resulted  in 
miss  distances  of  less  than  500  feet  slant  range  (See  Table  S-2) . 

An  analysis  of  these  seven  incidents  revealed  a  correlation  with 
individual  pilot  performance.  One  pilot  accounted  for  two  of  these 
seven  incidents,  and  the  other  five  pilots  also  exhibited  below 
average  miss  distances  in  their  other  encounters.  Conflict  geometry 
was  a  factor  in  resultant  miss  distance.  Fewer  close  conflicts 
occurred  while  the  CRT  display  presentation  was  being  used,  and 
more  conflict  situations  developed  when  the  IVSI  display  was  being 
used.  The  presence  of  traffic  information  was  cited  by  the  pilots 
and  test  observers  as  the  reason  for  this  difference  between  displays. 


Table  S-2.  ACHIEVED  MISS  DISTANCE 

Conflict 

Statistics 

Display  Type 

Total 

IVSI 

LED 

CRT 

Total  Number  of 

Miss  Distance 

Measurements 

191 

160 

165 

516 

Number  of  Close 

Encounters 
(<  *5  mile  in  range 
and  <  500'  in 
altitude) 

27 

28 

13 

68 

Percentage  of 

Close  Encounters 

to  Total  Encounters 

14 

18 

8 

13 

5 .  RECOMMENDATIONS 

The  pilots  who  participated  in  this  test  spontaneously  recommended  fur¬ 
ther  evaluation  and  demonstrations  that  should  be  accomplished  before  the 
implementation  of  an  ASA  system.  These  recommendations  cover  both  the  devel¬ 
opment  and  operational  implementation  of  the  ASA  program,  and  are  primarily 
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directed  toward  flight  evaluation  to  ensure  the  reliable  performance  of  the 
ASA  system  when  it  becomes  operational.  Specific  recommendations  are  dis¬ 
cussed  in  the  following  subsections. 

5 . 1  Flight  Test 

Consideration  should  be  given  to  both  operational  and  experimental 
flight  tests.  The  ASA  logic  should  be  subjected  to  an  operational  flight 
test  in  an  actual  airline  operational  environment  to  achieve  the  following 
objectives : 

•  Determine  the  incidence  of  ASA  alarms 

•  Determine  the  incidence  of  false  ASA  alarms 

•  Establish  the  sensitivity  of  ASA  performance  to  aircraft  density 

•  Identify  pilot  reactions  to  ASA  commands  in  actual  flight  conditions 

Experimental  flight  tests  should  be  conducted  by  the  FAA  to  stress  the  ASA 
logic  and  establish  the  satisfactory  operation  and  reliability  of  the  ASA 
concepts . 

5 . 2  ASA  Logic  Modifications 

The  ASA  logic  should  be  modified  to  provide  the  same  minimum  display 
time  for  limit  commands  as  provided  for  positive  and  negative  commands. 

In  addition,  specific  geometries,  such  as  the  "tail  chase"  geometry,  should 
be  investigated  further  to  determine  if  the  situation  might  be  improved 
by  procedural  instructions. 

5 . 3  ASA  Displays 

The  design  and  selection  of  the  best  ASA  display  is  dependent  on  the 
aircraft's  normal  operational  environment.  Specific  consideration  should 
be  given  to  display  location,  use  of  color,  inclusion  of  traffic  advisories, 
symbolic  representation  of  information,  and  combined  function  displays 
(normal  instrument  function  plus  ASA) . 

5 . 4  ATC  and  Flight  Procedures 

Before  implementation  of  an  ASA  system,  a  set  of  pilot  procedures 
should  be  developed  in  the  areas  of  response  to  ASA  traffic  advisories, 
response  to  ASA  commands,  and  communication  with  ATC  during  a  conflict 
situation.  A  training  program  should  be  established  to  explain  how  the 
system  functions,  what  each  type  of  command  advisory  represents  in  terms 
of  a  traffic  situation,  and  what  command  sequences  might  be  produced. 
Normally,  the  development  of  pilot  training  programs  and  flight  procedures 
is  the  responsibility  of  the  aircraft  operator,  subject  to  the  approval  of 
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the  FAA.  It  is  expected  that  pilot  procedures  will  be  developed  by  a  com¬ 
mittee  composed  of  airline  industry  personnel  and  approved  by  the  Federal 
Aviation  Administration. 
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GLOSSARY 


ACAS 

ALPA 

ASA 

ATA 

ATARS 

ATC 

ATCRBS 

BADCOM 


BCAS 

CAS 
CD  I 
CDTI 
CRT 

DABS 

EHSI 

FAA 

FDI 

FIM 

FL 

POI 

FPM 

F/E 

F/O 

GMN 

HSI 

HUD 

IAS 

IFR 


Airborne  Collision  Avoidance  System 
Air  Line  Pilots  Association 
Aircraft  Separation  Assurance 
Air  Transport  Association 

Automatic  Traffic  Advisory  and  Resolution  Service 
Air  Traffic  Control 

Air  Traffic  Control  Radar  Beacon  System 

Display  device  developed  by  the  Federal  Aviation  Administration 
that  uses  lighted  indicators  to  display  commands  and  relative 
intruder  position. 

Beacon  Collision  Avoidance  System 

Collision  Avoidance  System 
Course  Deviation  Indicator 
Cockpit  Display  of  Traffic  Information 
Cathode  Ray  Tube 

Discrete  Address  Beacon  System 

Electronic  Horizontal  Situation  Indicator 

Federal  Aviation  Administration 
Flight  Director  Indicator 
Fillmore  VOR 
Flight  Level 

Flight  Operations  Instructor 
Feet  Per  Minute 
Flight  Engineer 
First  Officer 

Gorman  VOR 

Horizontal  Situation  Indicator 
Head- Up  Display 

Indicated  Airspeed 
Instrument  Flight  Rules 
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ILS 

IMC 

Intruder 

IVSI 

KIAS 

LAX 

LED 

LMT 

MSL 

NAFEC 

NBAA 

OPR 

ORD 

PMD 

POM 

PWI 

SAN 

S  BA 

SFO 

SID 

SLI 

SMATC 

SOP 

STAR 

S/O 

Tau 


VFR 
VMC 
VOR 
VTU 
VS  I 

3 AT I  CRT 


Instrument  Landing  System 
Instrument  Meteorological  Conditions 

An  aircraft  which  violates  the  PWI  criteria  for  own  aircraft 
and  represents  a  potential  threat. 

Instantaneous  Vertical  Speed  Indicator 

Knots  Indicated  Airspeed 

Los  Angeles  International  Airport 

Light  Emitting  Diode 

Limit 

Mean  Sea  Level 

National  Aviation  Facilities  Experimental  Center 
National  Business  Aircraft  Association 

Operator 

Chicago  O' Hare  International  Airport 

Palmdale  VOR 
Pomona  VOR 

Proximity  Warning  Indication 

San  Diego  International  Airport 
Santa  Barbara  VOR 

San  Francisco  International  Airport 
Standard  Instrument  Departure 
Seal  Beach  VOR 

Short  Message  Air  Traffic  Control  Device 
Standard  Operating  Procedure 
Standard  Terminal  Arrival  Route 
Second  Officer 

A  derived  quantity  usually  expressed  in  seconds  which 
represents  the  time  to  closest  approach  of  two  aircraft. 

Tau  is  defined  as  range  divided  by  range  rate. 

Visual  Flight  Rules 

Visual  Meteorological  Conditions 

Very  High  Frequency  Omnidirectional  Range 

Ventura  VOR 

Vertical  Speed  Indicator 

Cathode  Ray  Tube  device  that  can  be  installed  in  a  standard 
3  inch  instrument  case. 
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CHAPTER  ONE 


INTRODUCTION 


1.1  PROJECT  OVERVIEW 

The  Systems  Research  and  Development  Service  of  the  Federal  Aviation 
Administration  (FAA)  is  conducting  a  program  to  develop  an  Aircraft  Sepa¬ 
ration  Assurance  (ASA)  system  for  use  within  the  National  Airspace  System 
(NAS) .  The  program  includes  examination  of  methods  for  detecting  potential 
aircraft  conflicts,  resolving  such  conflicts,  and  displaying  appropriate 
information  to  pilots  and  controllers.  Work  is  progressing  in  all  areas. 
This  report  provides  the  results  of  an  airline  cockpit  simulator  study  that 
was  designed  to  determine  the  cockpit  information  requirements  for  an  air¬ 
borne  collision  avoidance  system. 

The  study  involved  the  simulation  of  a  jet  transport  aircraft  equipped 
with  an  Aircraft  Separation  Assurance  System  operating  in  the  near-term 
traffic  environment  projected  for  a  major  air  transportation  hub.  The 
overall  objective  of  the  study  was  to  determine  the  cockpit  information 
requirements  for  an  airborne  collision  avoidance  system  within  a  realistic 
airline  operational  environment.  Specific  objectives  are  presented  in 
Section  1.2. 

Previous  simulations  of  ASA  systems  have  concentrated  on  light  air¬ 
craft  operated  by  one  pilot  in  a  one-on-one  encounter  (see  NAFEC  technical 
letter  report  NA-77-73-LR) .  This  ASA  simulation  was  unique  in  that  it 
used  a  jet  transport  cockpit  simulator  operated  by  qualified  flight  crews 
in  a  realistic  traffic  environment. 

The  selected  ASA  display  devices  covered  a  range  of  techniques  and 
capabilities.  One  device,  the  modified  Instantaneous  Vertical  Speed  Indi¬ 
cator  ( IVSI ) ,  combines  the  ASA  function  with  an  existing  instrument.  The 
other  displays  represent  new  technology  or  a  new  application  of  existing 
technology  to  a  cockpit  display. 

The  selected  ASA  collision  avoidance  concept  used  in  the  simulation 
was  the  Beacon  Collision  Avoidance  System  (BCAS)  developed  by  the  Federal 
Aviation  Administration*.  The  collision  avoidance  logic**  operates  in  two 


*FAA  BCAS  Concept,  Report  No.  FAA-EM-78-5,  April  1978. 

**Initial  Collision  Avoidance  Algorithms  for  the  Beacon-Based  Collision 
Avoidance  System,  Report  No.  FAA-RD-77-163  (as  revised) ,  April  1977. 
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modes.  The  first  mode,  which  is  analogous  to  the  logic  used  in  the  active 
BCAS  concept,  operates  on  range  and  altitude  information  from  other  air¬ 
craft.  This  mode  is  referred  to  in  this  report  as  active  mode  ASA.  The 
second  mode,  which  is  analogous  to  the  logic  used  by  the  full  BCAS  concept, 
uses  bearing  information  in  addition  to  range  and  altitude  and  is  referred 
to  as  full  mode  ASA. 

ARINC  Research  Corporation  conducted  this  airline  simulation  effort 
with  one  subcontractor.  United  Airlines.  United  Airlines  provided  a  flight 
simulator,  test  crews,  and  maintenance  support  for  the  simulation.  The 
work,  which  began  9  December  1977,  was  performed  under  Contract  DOT-FA78- 
4091.  The  overall  contract  effort  is  structured  into  Part  I,  Simulation; 
and  Part  IX,  Preparation  for  Operational  Testing.  This  report  describes 
Part  I.  A  separate  report*  describes  Part  II  and  provides  a  preliminary 
plan  for  the  operational  flight  test  of  the  Active  Beacon  Collision  Avoid¬ 
ance  System  (BCAS) . 

The  following  seven  tasks  constitute  the  Part  I  contractual  effort: 

1.  ASA  Simulation  Verification.  Working  jointly  with  the  Air  Trans¬ 
port  Association  (DABS/ATARS/BCAS  Task  Force)  and  FAA,  ARINC 
Research  Corporation  developed  conflict  geometries  and  scenarios 
that  represented  realistic  traffic  situations  and  exercised  the 
collision  avoidance  algorithms  to  the  full  extent. 

2.  Prepare  ASA  Simulator  Evaluation  Test  Plan.  A  Test  Plan**  was 
developed  to  define  the  specific  test  conditions  and  the  number 
and  type  of  experiments  performed. 

3.  Develop  ASA  Simulation  Control  Computer.  A  minicomputer  system 
was  procured  and  programmed  to  provide  an  ATC  environment  for  the 
simulation  test  bed.  The  minicomputer  was  also  used  to  provide 
the  interface  with  the  airline  simulator,  control  the  ASA  devices, 
and  collect  and  record  test  data. 

4.  Install  ASA  Simulation  Control  Computer  in  Simulation  Test  Bed. 

The  hardware  and  software  interface  requirements  were  defined,  and 
the  ASA  systems  simulator  was  integrated  into  the  test  bed. 

5.  Conduct  ASA  Concept  Evaluacion.  Various  ASA  concepts  developed 
by  the  DABS/ATARS/BCAS  Task  Force  and  FAA  were  evaluated,  with 
airline  pilots  being  utilized  as  subjects,  to  determine  the  impact 
of  these  concepts  in  the  cockpit  environment. 

6.  Analyze  Data  and  Prepare  Report.  Data  were  collected  and  analyzed. 
The  results  of  that  analysis  appear  in  this  report. 


*Preliminary  Plan  for  the  Operational  Flight  Test  of  the  Active  Beacon 
Collision  Avoidance  System,  ARINC  Research  Publication  1343-01-2-1936, 

June  1979. 

**Experimental  Test  Plan  for  the  Evaluation  of  Aircraft  Separation  Assurance 
System  Using  Airline  Flight  Simulators ,  ARINC  Research  Publication  1343- 
01-1-1753,  November  1978. 
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7.  Document  the  Simulation  Configuration.  A  description  of  the 
features  and  capabilities  of  the  experimental  test  bed  was 
developed.  It  is  included  in  this  report. 


1.2  OBJECTIVES 

The  primary  objective  of  the  experiment  was  to  determine  the  cockpit 
information  requirements  for  an  airborne  collision  avoidance  system.  A 
significant  secondary  objective  was  to  evaluate  the  operational  impact  of 
the  introduction  of  Aircraft  Separation  Assurance  Systems  in  commercial 
air  carrier  aircraft.  A  further  objective  was  to  expose  a  number  of  air 
carrier  pilots  to  the  ASA  concept  by  their  participation  in  the  experiment 
and  to  obtain  their  opinions  regarding  the  ASA  displays,  the  expected 
escape  maneuvers,  and  the  ASA  concept  as  currently  defined. 

The  candidate  ASA  display  devices  selected  for  use  in  the  simulation 
included  three  distinct  display  types:  (1)  an  existing  electromechanical 
device  (IVSI)  modified  to  display  ASA  commands  using  lighted  display  segments, 

(2)  an  alphanumeric  device  that  could  display  a  40-character  message  (using 
LED  technology)  to  provide  both  ASA  commands  and  traffic  advisories,  and 

(3)  a  CRT  device  that  combines  alphanumerics  and  symbols  to  provide  a  graph¬ 
ical  presentation  of  the  conflict  situation  and  an  alphanumeric  readout  of 
the  ASA  command.  These  candidate  displays  represented  selections  from  a 
larger  field  of  candidates  that  had  been  progressively  narrowed  and  redefined 
as  a  result  of  previous  related  experiments;  they  were  selected  in  coordina¬ 
tion  with  representatives  of  the  Air  Transport  Association,  the  Air  Line 
Pilots  Association,  and  the  FAA. 

Although  airline  pilots'  opinion  have  been  obtained  previously  on  a 
volunteer  basis,  this  is  the  first  time  a  relatively  large  number  of  pro¬ 
fessional  line-qualified  crews  have  evaluated  these  devices  in  standard- 
configuration  air-carrier  simulators.  Their  opinions  and  suggestions 
relating  to  operational  procedures  are  considered  invaluable. 


1.3  REPORT  ORGANIZATION 

This  report  is  organized  into  six  chapters  and  seven  appendixes. 

Chapter  Two  describes  the  experimental  approach  used  in  conducting  the 
evaluation.  Chapter  Three  is  a  discussion  of  the  experimental  design  and 
its  content.  Chapters  Four  and  Five  present  analyses  of  the  qualitative  and 
quantitative  data  collected,  respectively.  Chapter  Six  includes  conclusions 
drawn  from  the  analysis  and  recommendations  for  additional  testing  and 
system  implementation. 

The  following  appendixes  are  included  in  Volume  II  of  this  report: 

Appendix  A  -  Flight  Scenarios 

Appendix  B  -  Subject  Pilot  Advance  Briefing 

Appendix  C  -  Subject  Pilot  Pre-Briefing 
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Appendix  D  -  Flight  Crew  Questionnaire 

Appendix  E  -  Supplemental  Flight  Crew  Questionnaire 

Appendix  F  -  Data  Analysis  Formulas 

Appendix  G  -  Test  Bed  Description  Details 


CHAPTER  TWO 


EXPERIMENTAL  APPROACH 


In  this  chapter,  the  overall  evaluation  approach  is  discussed  and  the 
experimental  equipment  is  described.  The  discussion  is  intended  to  provide 
a  functional  overview  of  the  simulation  test  bed.  Specific  details  on  the 
test  bed  design  (e.g. ,  interface  techniques,  software  design)  are  included 
in  Appendix  G. 


2.1  OVERALL  APPROACH 

The  experimental  test  plan  describes  a  program  of  evaluation  using 
professional  flight  crews  in  a  United  Airlines  Boeing  727  cockpit  simulator. 
Six  different  missions  or  scenarios  of  about  45  minutes  each,  representing 
typical  airline  flights,  were  conducted  to  evaluate  the  ASA  concepts. 

Each  flight  crew  flew  three  of  the  six  scenarios  and  was  exposed  to  each 
of  the  three  ASA  displays.  Twenty-eight  crews  (consisting  of  volunteers 
recommended  by  the  Air  Line  Pilots  Association,  airline  management,  govern¬ 
ment,  and  industry)  participated  in  the  program.  Two—  or  three— subject 
crews  were  used,  for  a  total  of  74  participants.  Clearances  were  provided 
by  a  local  air  traffic  controller,  who  also  "provided  control"  for  other 
aircraft  in  the  scenario  to  create  a  realistic  communications  workload. 

The  simulator  was  equipped  with  a  computer-generated  visual  system  that 
projected  a  night  visual  scene  on  the  windshield  of  the  cockpit.  This 
system  was  modified  so  that  other  simulated  aircraft  would  appear  within 
the  scene  when  their  position  relative  to  the  simulator  cockpit  was  within 
the  forward  windshield  area. 

Both  qualitative  and  quantitative  data  were  collected  from  the  simu¬ 
lation.  Qualitative  data  were  developed  from  questionnaires  and  comments 
made  by  crew  members  and  test  observers.  Quantitative  data  were  collected 
by  a  minicomputer  system,  which  stored  data  on  simulator  position,  velocity, 
and  attitudes;  ASA  display  status;  and  unique  simulation  events. 

The  data  collected  were  grouped  and  analyzed  with  respect  to  influenc¬ 
ing  or  suspected  influencing  factors.  Appropriate  statistical  tests  were 
applied  to  the  results  of  this  analysis  to  identify  real  differences. 

On  the  basis  of  this  evaluation,  display  information  items  and  charac¬ 
teristics  have  been  ranked  qualitatively  and  quantitatively ,  the  impact  of 
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introducing  ASA  into  the  National  Airspace  System  has  been  discussed,  and 
additional  testing  has  been  recommended. 


2.2  SIMULATION  TEST  BED 

Figure  2-1  is  an  artist's  conception  of  the  simulation  test  bed  setup. 
This  section  describes  the  test  bed  —  an  interactive,  distributed  proc¬ 
essing  system  that  includes  a  simulator  cockpit  and  its  associated  control 
computer;  a  computer-generated  image  display  system;  a  simulation  control 
computer;  a  control/monitor  station;  and  associated  communications,  input/ 
output  devices,  and  data  storage  devices.  Omitted  from  this  discussion 
are  the  simulation  cockpit  displays,  which  are  described  in  Section  2.3. 

The  test  bed  was  designed  to  permit  evaluation  of  flight  crew  reactions 
to  aircraft  conflict  and  near-conflict  situations  that  could  present  a 
significant  element  of  danger  if  conducted  in  actual  aircraft.  The  use  of 
a  simulator  makes  experimental  problem  control  and  data  collection  much 
simpler  than  would  be  possible  in  an  e^qoeriment  using  actual  aircraft.  In 
addition,  simulation  presents  significant  cost  and  time  advantages  over  an 
experiment  that  might  require  several  transport  aircraft  operating  in  con¬ 
trolled  airspace  under  the  precise  conditions  necessary  for  experimental 
work. 

2.2.1  Aircraft  Simulator 

The  aircraft  simulator  selected  for  this  experiment  represents  the 
Boeing  727  aircraft,  one  of  the  most  widely  used  commercial  jet  transports. 
The  B-727  is  expected  to  remain  in  the  fleets  of  most  major  airlines  for 
the  next  20  years;  therefore,  its  performance  and  handling  characteristics 
make  it  most  appropriate  for  use  in  evaluating  ASA  displays. 

The  simulator  is  located  in  the  United  Airlines  Flight  Training  Center 
at  Stapleton  International  Airport,  Denver,  Colorado.  Its  cockpit  repli¬ 
cates  a  B-727  aircraft  in  the  United  Airlines  fleet.  The  cockpit  module 
layout  is  shown  in  Figure  2-2. 

Figures  2-3  and  2-4  show  the  captain's  and  second  officer's  instrument 
panels,  respectively.  Figure  2-5  presents  a  detail  of  the  center  console 
forward  of  the  throttle  quadrant  and  aft  of  the  instrument  panel.  The  ASA 
display  locations  are  included  in  the  diagrams  for  reference. 

The  cockpit  module  is  mounted  on  a  3-degrees -of- freedom  motion  base 
that  simulates  the  movement  of  an  aircraft  cockpit.  The  motion  base  and 
the  aircraft  instruments  are  controlled  by  a  GP-4  computer,  which  is  sup¬ 
plied  as  part  of  the  Singer-Link-developed  simulator  and  is  located  in  a 
room  adjacent  to  the  cockpit  module.  Simulator  control  is  effected  through 
a  control  panel  located  inside  the  cockpit  module  at  the  second  officer's 
station.  This  panel  is  operated  by  the  flight  operations  instructor  (FOI) , 
who  also  served  as  the  second  officer  during  regular  training. 


2-2 


The  normal  mission  of  the  simulator  is  recurrent  and  upgrade  training 
of  United  Airlines  flight  crews  and  flight  crews  of  other  airlines,  Govern¬ 
ment  agencies,  and  private  operators  undergoing  training  at  the  center. 

The  simulator  is  designed  to  allow  the  flight  operations  instructor  to 
introduce  adverse  weather,  turbulence,  and  system-malfunction  problems  to 
the  crew  undergoing  training.  It  can  create  catastrophic  conditions  in  an 
extremely  realistic  manner. 

2.2.2  Computer-Generated  Image  (CGI)  System 

A  computer-generated  scene,  depicting  a  nighttime  view  from  the  for¬ 
ward  windshield  of  a  jet  transport  aircraft  is  produced  by  a  separate  com¬ 
puter  system  (also  developed  by  Singer-Link)  and  is  projected  on  the  forward 
windows  of  the  simulator  to  add  realism  to  the  regular  flight  training 
program  (see  Figure  2-6) .  The  scene  consists  of  light  points  that  provide 
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Figure  2-6.  COMPUTER-GENERATED  NIGHT  VISUAL  SCENE 


a  full-color  representation  of  a  night  landscape  for  a  large  metropolitan 
area,  including  that  city's  major  airport. 

The  landscape  scene,  consisting  of  more  than  7,000  lights,  is  created 
from  a  data  base  of  15,000  lights.  This  data  base  is  stored  in  the  CGI 
computer  as  a  file  of  3-dimensional  coordinates  characterized  as  to  color 
and  intensity.  Simulator  position  and  attitude  are  obtained  through  an 
interface  to  the  GP-4  computer.  The  displayed  lights  are  those  which  fall 
into  the  forward  field  of  view,  which  is  approximately  45°  right,  55°  left, 
30°  up,  and  60°  down.  Additional  controls  are  available  on  the  simulator 
OGI  control  panel  (Figure  2-2)  to  simulate  various  visibilities,  ceiling 
heights,  cloud  tops,  and  ground  fog. 

For  purposes  of  this  experiment,  the  programming  of  the  OGI  computer 
was  modified  so  that  other  aircraft  could  be  simulated  in  the  visual  scene. 
Each  aircraft  was  represented  by  a  pair  of  lights  consisting  of  a  flashing 
red  and  a  flashing  white  light  that  were  spaced  60  feet  apart.  Those 
simulated  aircraft,  whose  relative  position  to  the  simulator  fell  within 
the  forward  viewing  area,  were  displayed  within  the  visual  scene.  Data  for 
the  other  aircraft  were  obtained  through  an  interface  to  the  simulation 
control  computer  (Subsection  2.2.3).  The  position  information  was  updated 
20  times  per  second  to  provide  display  continuity.  The  red  and  white  lights 
were  processed  independently  to  provide  proper  viewing  perspective.  In  all 
other  respects,  the  moving  lights  representing  the  other  aircraft  were 
controlled  in  the  same  manner  as  the  fixed  ground  lights. 

2.2.3  Simulation  Control  Computer 

The  simulation  control  computer  produced  the  ATC  environment  for  the 
simulation  test  bed  and  was  used  to  control  simulation  activities.  The  ATC 
environment  software  consisted  of  the  following  features: 

•  Traffic  and  intruder  generation  logic 

•  Simulator  cockpit  interface 

•  Alpha/beta  tracker 

•  Detection  and  resolution  logic 

•  Cockpit  display  logic 

•  ATC  displays 

The  simulation  control  features  included  simulation  initialization,  simu¬ 
lation  interaction  logic,  and  data  recording  algorithms. 

The  simulation  control  computer  was  a  Government-provided  Digital 
Equipment  Corporation  PDP-11/34  minicomputer.  The  minicomputer  was  inter¬ 
faced  directly  to  the  United  Airlines  CGI  computer  by  using  a  Digital 
Equipment  Corporation  UNIBUS  window.  The  UNIBUS  window  provides  an  inter¬ 
processor  link  between  computers  through  a  common  UNIBUS.  Direct  memory 
access  into  the  "other"  computer's  memory  is  efficiently  performed  by 
using  "cycle  stealing"  techniques. 
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The  interface  techniques  used  and  the  software  design  are  described 
in  Appendix  G. 

2.2.4  Control/Monitor  Station 

The  control/monitor  station  (Figure  2-7)  served  two  functions:  simu¬ 
lation  control  and  monitoring,  and  ATC.  These  functions  were  provided  by 
two  input/output  devices  interfaced  directly  to  the  simulation  control 
computer.  A  video  terminal  was  used  by  the  simulation  controller  (Sub¬ 
section  3.5.2)  to  display  information  of  transitory  interest  and  to  inter¬ 
act  with  the  simulation.  A  storage  tube  graphic  display  terminal  was  used 
by  the  air  traffic  controller  (Subsection  3.5.3)  to  display  a  history  of 
the  simulator  and  aircraft  tracks  during  the  conduct  of  the  simulation. 

2.2.5  Communications  System 

A  private  communications  link  (sound-powered  headsets)  connected  the 
simulation  controller  with  the  test  observer.  The  ATC  controller  had  direct 
access  to  the  simulator  interphone  systems  for  the  purpose  of  issuing  ATC 
instructions.  Examples  of  the  types  of  communications  are  discussed  in 
Chapter  Three. 


Figure  2-7.  CONTROL/MONITOR  STATION 


2.2.6  Data  Recording  Equipment 


Several  different  media  were  used  to  record  data.  Simulator  parameters 
(position,  velocity,  and  attitude) ,  command  and  intruder  data,  and  simula¬ 
tion  controller  inputs  were  recorded  on  high-speed  disks  and  later  trans¬ 
ferred  to  magnetic  tape  for  more  permanent  storage.  Pilot  opinion  was 
recorded  on  questionnaires.  Cassette  tape  recorders  were  used  to  record 
cockpit  chatter,  selected  observer  comments,  and  discussion  during  a 
debriefing  session. 


2.3  SIMULATION  COCKPIT  DISPLAYS 

The  test  displays  selected  for  use  in  the  simulation  represent  three 
distinct  methods  for  presenting  ASA  information  to  the  flight  crew.  Each 
method  corresponds  to  a  different  display  technique  and  a  different  level 
of  ASA  information.  The  selected  test  displays  are  as  follows: 

•  Modified  Instantaneous  Vertical  Speed  Indicator  (IVSI) 

•  Light  Emitting  Diode  (LED)  Display 

•  Cathode  Ray  Tube  (CRT)  Display 

The  modified  IVSI  uses  lighted  indicators  to  display  ASA  commands. 

The  LED  display  presents  ASA  commands  and  traffic  advisories  in  alphanumeric 
form  using  LED  technology.  The  CRT  display  provides  a  graphical  presenta¬ 
tion  of  ASA  commands  and  traffic  advisories.  An  audio  alert,  common  to 
all  displays,  is  sounded  whenever  a  new  ASA  command  is  displayed. 

All  of  the  selected  displays  can  be  used  with  either  an  active  or 
full  mode  ASA  system  or  a  combination  system.  In  the  simulation  test  bed, 
the  displays  were  driven  by  the  simulation  control  computer,  which  tracked 
the  position  of  the  simulator  and  all  scenario  aircraft,  applied  the  ASA 
detection  and  resolution  logic,  determined  the  appropriate  advisory  or 
warning  message,  formatted  the  message  for  thr  display  under  test,  and 
transmitted  the  message  to  the  cockpit  display. 

The  ASA  logic  operates  in  the  following  conceptual  manner.  The  logic 
defines  a  protected  volume  of  airspace  around  each  aircraft  and  compares 
aircraft  positions  to  determine  if  that  airspace  has  been  violated.  The 
criteria  for  establishing  the  protected  volumes  include  range,  range  rate, 
and  tau  (range  divided  by  range  rate)  in  both  the  horizontal  and  vertical 
planes.  Two  volumes  are  established.  A  larger  volume  is  used  for  deter¬ 
mining  potential  threatening  aircraft.  This  logic  is  often  called  pilot 
warning  indication  (PWI)  logic  and  causes  the  display  of  traffic  advisory 
information.  Violating  the  smaller  protected  volume  requires  action  by 
the  aircraft  involved  and  causes  the  display  of  ASA  commands. 

The  displays  used  represent  a  good  engineering  design;  however,  opti¬ 
mization  of  the  displays  was  outside  the  scope  of  this  simulation.  Comments 
received  from  the  test  crews  have  provided  some  insight  into  features  that 
should  be  included  in  any  optimized  display. 
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The  display  methods  used  were  selected  from  a  larger  set  (Table  2-1) 
in  coordination  with  pilot,  airline,  and  Government  committees.  Seven 
types  of  displays  were  presented.  The  recommended  selection  criteria 
were  as  follows: 

•  Each  display  should  contain  a  different  level  of  information  content. 

•  Each  display  should  represent  a  different  technological  approach. 

•  All  displays  should  be  operated  by  the  same  ASA  logic. 

An  additional  factor  considered  was  the  installation  of  the  displays  into 
the  cockpit  simulator.  The  simulator  could  not  be  dedicated  to  the  program 
on  a  full-time  basis  and  had  to  be  returned  to  its  certified  configuration 
after  each  test  session.  In  addition,  major  modifications  could  not  be 
made  to  the  instrument  panels.  Therefore,  the  selected  displays  would  have 
to  replace  existing  instruments  and  meet  their  size  specifications.  This 
was  not  a  serious  limitation,  since  similar  consideration  will  be  necessary 
for  actual  implementation. 


Table  2-1.  ASA  DISPLAY  FUNCTIONAL  ASSESSMENT 

Displays 

Vertical 

Commands 

Horizontal 

Commands 

Traffic 

Situation 

Proximity 

Warning 

Indication 

(PWI) 

Limit 

Vertical 

Limit 

Horizontal 

1VSI 

Yes 

Requires 

Modification 

No 

Perhaps  with 
Modification 

Yes 

Yes 

SMATC 

Yes 

Yes 

No 

Limited 

Yes 

Yes 

BADCOM 

Yes 

Yes 

No 

Yes 

Yes 

Yes 

CDTI 

Symbology 

On  ly 

No 

Yes 

Yes 

Yes 

No 

Yes 

CRT 

Alphanumeric 

and 

Symbology 

Yes 

Yes 

No 

Limited 

Yes 

Yes 

Yes 

3ATI 

CRT 

Yes 

Yes 

No 

Limited 

Yes 

Yes 

Yes 

LED 

Alphanumeric 

and 

Symbology 

Yes 

Yes 

No 

Yes 

Yes 

Yes 
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2.3.1  Modified  Instantaneous  Vertical  Speed  Indicator  (IVSI) 

The  modified  IVSI  display  is  the  only  test  display  that  combines  the 
ASA  function  with  another  display  function.  The  test  display  replaced  the 
existing  IVSI  on  both  the  captain's  and  first  officer's  instrument  panels 
(Figures  2-3  and  2-4) .  Vertical  speed  on  the  test  display  is  indicated  in 
the  same  manner  as  on  the  standard  IVSI. 

Modification  of  the  standard  IVSI  to  allow  presentation  of  ASA  com¬ 
mands  consisted  of  the  following  additions  (see  Figure  2-8  for  an  artist's 
rendition  of  the  display) : 

•  Lighted  red  arrows  for  alt  positive  commands 

•  Lighted  yellow  indicators  for  negative  horizontal  commands 

•  Lighted  yellow  arc  segments  for  limit  vertical  commands 

The  positive  command  arrows  instruct  the  pilot  to  climb,  descend, 
turn  right,  or  turn  left;  they  are  labeled  "climb",  "dive",  "right",  and 
"left",  respectively.  (See  Figure  2-9A  for  a  photograph  of  the  IVSI  illus¬ 
trating  a  descend  command.)  The  negative  horizontal  command  indicators 
instruct  the  pilot  to  stop  turning  right  or  left  and  are  labeled  "no 
right"  and  "no  left."  The  limit,  vertical  command  lighted  arc  segments  are 
unlabeled  and  instruct  the  pilot  to  limit  his  climb  or  descent  to  500, 
1,000,  or  2,000  feet  per  minute  ( FPM )  (see  Figure  2-9b  for  a  photograph 


Figure  2-8.  ARTIST'S  DRAWING  OF 
MODIFIED  IVSI 
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Figure  2-9a.  IVSI  ILLUSTRATING  DESCENT  COMMAND 


Figure  2-9b.  IVSI  ILLUSTRATING  LIMIT  DESCENT  COMMAND 
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of  the  IVSI  illustrating  a  limit-descent  command  to  500  FPM .  The  lighted 
arc  segments  are  used  in  combination  to  produce  the  desired  command.  A 
single  segment  between  the  2,000  FPM  and  6,000  FPM  positions  on  the  IVSI 
is  lighted  to  indicate  a  limit-climb  command  or  limit-descent  command  of 
2,000  FPM.  Two  segments,  one  between  the  1,000  FPM  and  2,000  FPM  positions 
and  one  between  the  2,000  FPM  and  6,000  FPM  positions,  are  lighted  to  indi¬ 
cate  a  limit  climb  or  descent  to  1,000  FPM.  The  lighting  of  all  three 
segments  indicates  a  command  to  limit  climb  or  descent  to  500  FPM.  To 
reduce  confusion,  pilots  were  instructed  simply  to  keep  the  vertical  speed 
indicator  out  of  the  lighted  arc  segments. 

Only  one  command  can  be  displayed  at  any  time,  and  the  display  remains 
lighted  for  the  duration  of  the  command.  At  the  end  of  the  command,  the 
display  is  cleared. 

2.3.2  LED  Display 

The  LED  display  used  in  the  ASA  simulation  represents  the  first  appli¬ 
cation  of  this  technology  in  an  air  transport  aircraft  cockpit.  The  device 
is  a  three-color  display  that  provides  traffic  advisory  messages  and  ASA 
commands  in  alphanumeric  characters,  augmented  by  a  limited  number  of 
symbols.  The  traffic  advisory  messages  are  presented  in  a  shorthand  nota¬ 
tion  similar  to  that  which  is  used  by  many  pilots.  Two  LED  displays  were 
installed,  one  on  the  captain's  instrument  panel  and  one  on  the  first 
officer's  instrument  panel  (see  Figures  2-3  and  2-4). 

The  LED  display  (see  Figure  2-10)  is  capable  of  producing  up  to  40 
characters  or  symbols  arranged  in  4  lines  of  10  characters  each.  However, 
because  of  parallax  problems  caused  by  display  position,  only  the  3  lower 
lines  were  usable.  Each  line  of  text  is  color-coded  according  to  the  type 
of  message  that  is  being  displayed.  Traffic  advisories  are  displayed  in 
green,  negative  and  limit  vertical  commands  in  amber,  and  positive  commands 
in  red. 

The  traffic  advisories  are  abbreviated  into  a  10-character  message 
consisting  of  intruder  bearing,  range,  heading,  and  altitude*.  The  bearing 
is  represented  by  a  clock  position  (1  to  12)  followed  by  a  degree  (°)  indi¬ 
cation.  Range  is  displayed  as  a  two-character,  left- justified  expression 
in  nautical  miles.  The  intruder  heading  is  truncated  into  one  of  the 
eight  cardinal  directions  and  displayed  as  a  two-character  expression. 
Altitude  is  expressed  in  hundreds  of  feet  Mean  Sea  Level  (MSL)  in  a  three- 
character  field.  For  example.  Figure  2-10  indicates  traffic  at  12  o'clock, 
2  miles  away,  southeastbound,  at  6,100  feet  MSL.  In  the  experiment, 
advisories  were  presented  when  an  aircraft  violated  the  ASA  PWI  criteria. 

Up  to  three  traffic  advisories  can  be  displayed  concurrently  in  non¬ 
conflict  situations  (i.e.,  the  traffic  situation  is  such  that  an  ASA 
command  is  not  required).  Advisories  are  displayed  on  lines  2,  3,  and  4 
starting  with  the  bottom  line.  The  list  is  scrolled  down  when  an  advisory 
disappears. 


*In  Active  Mode,  bearing  and  heading  were  not  displayed. 
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Figure  2-10.  LED  DISPLAY 


When  an  ASA  command  is  required,  a  single  traffic  advisory  (for  the 
aircraft  involved  in  the  conflict)  is  presented  on  line  4,  with  the  command 
displayed  on  lines  2  and  3.  The  command  formats  appear  in  Table  2-2. 

Only  one  command  can  be  displayed  at  any  time,  and  the  display 
remains  lighted  for  the  duration  of  the  command.  When  the  command  is 
deleted,  the  display  is  again  capable  of  showing  as  many  as  three  traffic 
advisories.  If  there  are  no  required  traffic  advisories,  the  display  is 
cleared. 

The  LED  display  was  developed  by  the  Aero  Products  Division  of  Litton 
Systems,  Inc.,  who  provided  it  for  use  in  this  project.  Their  interest, 
advice,  and  assistance  have  been  greatly  appreciated. 

2.3.3  CRT  Display 

The  CRT  display  combines  symbology  and  alphanumerics  to  create  a 
graphical  presentation  of  the  conflict  situation.  The  display  was  installed 
in  the  forward  pedestal  area  of  the  simulator  in  place  of  the  weather 
radar,  which  is  not  used  in  normal  simulator  training  (Figure  2-5) .  The 
display  is  heading-oriented  and  own-aircraft-centered  to  present  the  same 
view  as  that  which  the  pilot  sees  out  of  the  cockpit  window.  The  full  mode 
presentation  of  the  display  is  illustrated  in  Figure  2-11. 


Table  2-2.  PRESENTATION  OF  COMMANDS  ON 
LED  DISPLAY 


Command 


CLIMB 


Presentation  Color 


+'H'  RED 

CLIMB 


DESCEND 


DESCEND 


TURN 

RIGHT 


DON'T 

CLIMB 


DON'T 

CLIMB 


AMBER 


DON'T 

DESCENT 


DON'T 

DESCEND 


AMBER 


DON'T 

TURN  RIGHT 

DON'T 

TURN  RIGHT 

AMBER 

DON'T 

TURN  LEFT 

DON’T 

TURN  LEFT 

AMBER 

LIMIT  CLIMB 

500  FPM 

LMT  500 

CLIMB 

AMBER 

LIMIT  CLIMB 

1000  FPM 

LMT  1000 

CLIMB 

AMBER 

LIMIT  CLIMB 

2000  FPM 

LMT  2000 

CLIMB 

AMBER 

LIMIT  DESCENT 

500  FPM 

LMT  500 
DESCENT 

AMBER 

LIMIT  DESCENT 
1000  FPM 

LMT  1000 
DESCENT 

AMBER 

LIMIT  DESCENT 

2000  FPM 

LMT  2000 

DESCENT 

AMBER 
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DESCEND 


Oii'Jl'J 


liMSiU 


Own-aircraft  is  symbolized  by  a  small  aircraft  symbol  positioned  in 
the  center  horizontally  and  positioned  two-thirds  down  from  the  top 
vertically.  A  numerical  data  block  positioned  to  the  left  of  the  symbol 
provides  own-aircraft  altitude  in  hundreds  of  feet  MSL.  The  aircraft 
symbol  is  surrounded  by  range  rings  that  correspond  to  ranges  of  3  and  6 
miles. 

Aircraft  that  violate  the  ASA  PWI  logic  and  are  within  the  range 
limitations  of  the  display  (6  miles  on  either  side  or  behind,  9  miles 
ahead)  are  symbolized  by  a  ring  of  dots.  Present  and  previous  aircraft 
position  is  represented  by  a  trail  of  dots  that  begins  at  the  center  of 
the  aircraft  symbol.  The  trail  or  history  corresponds  to  the  actual  posi¬ 
tion  in  space  that  the  aircraft  occupied  relative  to  own-aircraft  present 
position.  To  the  right  of  the  aircraft  symbol  is  a  numerical  data  block 
containing  the  aircraft's  altitude  in  hundreds  of  feet  and  altitude  trend. 
This  trend  is  represented  by  an  arrow  for  climbing  or  descending  aircraft; 
when  the  aircraft  is  level,  there  is  no  symbol. 

Intruder  aircraft  (i.e.,  those  aircraft  which  violate  the  ASA  detec¬ 
tion  logic  and  cause  an  ASA  command  to  appear)  are  displayed  in  the  same 
manner  as  other  aircraft  with  the  exception  that  the  aircraft  symbol  is 
filled  in. 

ASA  commands  are  displayed  in  large  alphanumerics  at  the  bottom  of 
the  screen.  The  commands  are  formatted  on  a  single  line  by  using  the  same 
terminology  and  abbreviations  used  on  the  LED  (Table  2-2).  Arrows  are 
not  part  of  the  positive  command  presentation,  however. 

If  there  are  no  required  traffic  advisories  or  commands,  the  display 
consists  of  the  own-aircraft  symbol  and  associated  data  block  and  the  two 
aircraft  range  rings. 

In  active  mode,  since  only  range  and  altitude  information  is  available, 
an  entirely  different  display  presentation  is  required  (see  Figure  2-12) . 

An  alphanumeric  presentation  was  used;  it  consisted  of  a  heading  declaring 
the  ASA  mode  of  operation  and  specifying  that  displayed  data  will  consist 
of  range  and  altitude  of  an  intruder  aircraft.  Traffic  advisories  are 
presented  for  intruder  aircraft  only.  Range  is  displayed  in  nautical 
miles  and  altitude  is  in  hundreds  of  feet  MSL.  Commands  are  presented  in 
the  same  manner  as  previously  described.  The  rings  on  the  display  were 
not  used  during  active  mode  but  were  present  because  they  were  painted  on 
the  display  face;  however,  because  of  the  low  ambient  light  level  in  the 
cockpit  simulator,  they  were  less  prominent  than  they  appear  in  the 
photographs . 
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Figure  2-12.  CRT  DISPLAY  -  ACTIVE  MODE 

2-18 


a 


CHAPTER  THREE 


DESIGN  AND  CONDUCT  OF  EXPERIMENT 


The  overall  evaluation  objectives  are  expanded  into  specific  objectives 
in  this  section.  The  development  of  an  experimental  test  was  based  on 
these  specific  objectives  and  required  measurements  of  the  effects  of 
various  factors  on  pilot  performance  in  a  conflict  situation.  The  procedure 
used  to  acquire  experimental  pilots  and  the  characteristics  of  these  pilots 
are  discussed,  and  the  experimental  procedure  is  described. 


3.1  SPECIFIC  EVALUATION  OBJECTIVES 

The  overall  evaluation  objectives  stated  in  Chapter  One  were  further 
defined  in  terms  of  specific  objectives: 

•  To  rank  and  determine  the  criticality  of  specific  information 
elements  that  could  be  part  of  a  collision  avoidance  display. 

•  To  determine  pilot  reactions  to  three  different  techniques  of  dis¬ 
playing  collision  avoidance  information  in  a  cockpit  using  a  wide 
range  of  commercial  airline  pilots  and  to  rank  the  three  display 
techniques  on  the  basis  of  both  quantitative  and  qualitative  data. 

•  To  determine  the  impact  of  an  Aircraft  Separation  Assurance  System 
on  ATC  and  flight  procedures 

•  To  evaluate  the  effectiveness  of  ASA  commands  for  collision 
avoidance. 

•  'To  measure  and  record  the  response  characteristics  of  the  subject 
pilots  to  collision  avoidance  commands 

•  To  investigate  the  possible  effects  of  scenario,  display,  and 
experience  on  pilot  opinion 

To  meet  these  objectives,  both  quantitative  and  qualitative  data  were 
collected.  The  quantitative  data  were  collected  by  data  collection 
algorithms  in  the  simulation  control  computer.  Response  characteristics 
(response  time  and  magnitude  of  response) ,  generated  commands,  and  simu¬ 
lator  position,  velocity,  and  attitude  were  the  quantifiable  variables. 

The  qualitative  data  were  collected  from  the  debriefing  questionnaire, 
observations  of  the  test  observer,  and  the  supplemental  questionnaire. 


3.2  EXPERIMENTAL  DESIGN 

The  major  constraint  on  the  experimental  design  was  availability  of 
simulator  time.  Only  60  hours  of  simulator  time  was  available  for  the 
experiment.  A  two-hour  session  was  required  to  provide  sufficient  time 
for  each  crew  to  fly  a  scenario  that  included  all  phases  of  flight  with 
each  of  the  three  displays.  An  analysis  of  sample  size  and  sample  error 
showed  that  30  crews  would  provide  a  95  percent  confidence  level  in  the 
results  of  the  experiment. 

During  each  2-hour  simulator  session,  each  crew  flew  three  scenarios, 
using  a  different  ASA  display  for  each  scenario.  The  scenarios  were 
selected  from  a  pool  of  six  scenarios,  which  included  three  westbound  and 
three  eastbound  departures  and  arrivals.  Each  scenario,  as  described  in 
Appendix  A,  consists  of  a  flight  profile  typical  of  a  departure  from  the 
Los  Angeles  International  Airport  to  an  assigned  cruise  altitude,  and  a 
descent,  approach,  and  landing  at  the  same  airport.  Los  Angeles  was 
chosen  because  of  its  availability  as  a  data  base  for  the  computer-generated 
visual  scene  and  its  wide  use  as  the  operation  area  against  which  other  ASA 
studies  have  focused.  However,  the  scenario  traffic  densities  did  not  repre¬ 
sent  maximum  current  or  projected  traffic  densities  for  the  entire  Los 
Angeles  airspace. 

The  display/scenario  combinations  were  assigned  such  that  each  scenario 
was  flown  with  each  display  an  equal  number  of  times.  Further,  the  order 
of  presentation  of  the  scenarios  and  the  order  in  which  the  displays  were 
presented  were  evenly  distributed.  The  crew  test  assignments  appear  in 
Table  3-1. 

The  crews  used  were  volunteers  from  many  of  the  major  airlines,  govern¬ 
ment,  and  industry.  While  the  original  design  called  for  two  subject 
pilots  per  crew,  a  large  response  from  the  pilot  population  enabled  us  to 
use  three  subject  pilots  in  more  than  half  of  the  experimental  sessions. 

The  scenarios  consisted  of  six  conflicts  each,  the  maximum  number  of 
conflicts  that  could  reasonably  be  fitted  into  a  single  scenario  without 
overloading  the  pilot.  Each  conflict  represents  a  different  conflict 
geometry. 


3.3  EXPERIMENTAL  CREWS 

A  written  request  was  submitted  to  the  Air  Line  Pilots  Association 
(ALPA) ,  Air  Transport  Association  (ATA) ,  and  National  Business  Aircraft 
Association  (NBAA)  for  volunteers  to  participate  in  the  experiment.  The 
request  suggested  that  pilots  be  thoroughly  familiar  with  the  B-727  and 
preferably  flying  the  B-727  regularly  in  scheduled  service.  The  74  pilots 
that  participated  in  the  program  are  currently  flying  for  or  are  on  the 
staffs  of  the  following  organizations  (the  number  of  volunteers  from  the 
organization  appears  in  parentheses) : 

American  Airlines  (7) 
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Continental  Airlines  (7) 

Federal  Aviation  Administration  (3) 

Frontier  Airlines  (1) 

National  Aeronautics  and  Space  Administration  (2) 
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•  Pacific  Southwest  Airlines  (1) 

•  Pan  American  World  Airways  (2) 

•  Piedmont  Aviation  (4) 

•  Trans  World  Airlines  (6) 

•  United  Airlines  (31) 

•  United  Technologies  (2) 

•  Western  Airlines  (8) 

The  participating  pilots  could  be  classified  as  follows: 

•  Captain  (35) 

•  First  Officer  (21) 

•  Second  Officer  (5) 

•  Management  (6) 

•  Other  (7) 

Of  the  74  pilots,  58  are  currently  B-727- rated.  The  remaining  16  (all 
previously  B-727-qualified)  are  currently  flying  B-737,  B-747,  DC-8,  or 
DC-10  aircraft.  The  pilots  had  an  average  of  more  than  10,000  hours' 
flying  experience  of  which  500  hours  were  flown  in  the  past  year.  Each 
pilot  received  an  advance  briefing  packet  (Appendix  B)  to  familarize  him¬ 
self  with  the  experiment. 


3.4  SCENARIOS 

The  scenarios  were  selected  for  this  experiment  to  permit  evaluation 
of  crew  reaction  in  a  realistic  flight  situation.  Each  scenario  includes 
all  phases  of  flight:  departure,  climb,  en  route,  descent,  and  approach. 

Six  scenarios  were  used,  with  flights  departing  Los  Angeles  International 
Airport  using  standard  ATC  procedures  and,  after  a  mid-course  turn-around, 
receiving  clearance  back  to  Los  Angeles  for  an  approach  and  landing.  Each 
of  the  six  scenarios  describes  a  flight  of  about  35  minutes.  The  flight 
plan  information  was  presented  to  the  crew  before  the  experiment  started. 

Background  traffic  was  included  in  each  scenario  to  provide  a  realistic 
environment  in  the  vicinity  of  the  cockpit  simulator.  The  background  traf¬ 
fic  used  in  the  simulation  was  not  intended  to  represent  maximum  current  or 
projected  traffic  densities  in  the  entire  Los  Angeles  area.  These  aircraft 
were  preprogrammed  to  fly  typical  IFR  (Instrument  Flight  Rules)  and  VFR 
(Visual  Flight  Rules)  flight  paths  around  Los  Angeles  airport.  Altitudes 
and  flight  paths  were  set  so  that  some  of  the  aircraft  would  appear  in  the 
computer-generated  visual  scene.  While  conflicts  between  the  simulator 
and  the  background  aircraft  were  not  planned,  some  did  occur  as  a  result 
of  variations  in  the  way  the  subject  pilots  flew  the  simulator  and  responded 
to  commands.  Generally,  situations  developed  where  the  background  aircraft 


3-4 


was  visible  in  the  cockpit  visual  scene  but  was  not  considered  a  threat. 
Traffic  advisories  were  sometimes  presented  on  background  aircraft  without 
a  subsequent  command.  This  type  of  environment  minimized  the  likelihood 
of  the  crews  immediately  interpreting  each  aircraft  displayed  as  a  threat. 

Six  intruders  were  encountered  in  each  flight,  each  representing  a 
different  conflict  geometry.  A  similar  set  of  conflict  geometries  was 
used  in  all  flights;  however,  the  sequence  in  which  they  were  presented 
was  varied  to  realistically  accommodate  the  specific  flight  path.  The 
intruders  were  not  equipped  with  ASA  displays  and  therefore  did  not  respond 
to  the  conflict  situation.  The  scenarios  are  described  briefly  in  the  next 
few  sections  and  in  detail  in  Appendix  A.  The  specific  conflict  geometries 
used  also  appear  in  Appendix  A. 

3.4.1  Scenario  1:  Los  Angeles  to  Las  Vegas 

After  departure  on  runway  25R,  the  simulator  is  vectored  westbound 
over  the  ocean  until  it  reaches  6,000  feet,  where  it  is  turned  to  the 
departure  track  and  will  intercept  V210  to  Las  Vegas  in  the  vicinity  of 
Meant  Intersection.  The  first  conflict  occurs  in  the  turn,  with  an  air¬ 
craft  heading  eastbound  on  V299  descending  through  the  simulator's  cleared 
altitude  of  6,000  feet. 

Following  resolution  of  this  conflict,  the  simulator  is  vectored 
toward  Meant  and  cleared  to  climb  to  its  cruising  altitude  of  FL190.  During 
the  climb,  two  additional  conflicts  are  presented.  When  the  simulator 
arrives  over  Meant,  it  is  vectored  into  the  profile  approach  for  runway 
24R.  One  conflict  is  presented  during  the  vector,  and  two  additional  con¬ 
flicts  are  presented  on  the  final  approach.  One  of  these  conflicts 
simulates  an  overshoot  on  a  parallel  ILS  (Instrument  Landing  System) 
approach . 

3.4.2  Scenario  2;  Los  Angeles  to  San  Diego 

The  simulator  departs  runway  6L  and  climbs  to  4,000  feet.  A  head-on 
conflict  develops  with  an  intruder  on  the  24R  localizer,  level  at  4000 
feet.  The  simulator  is  vectored  direct  to  Seal  Beach  VOR  (SLI) ,  departing 
on  a  148°  radial  to  intercept  V25  and  cleared  to  its  cruising  altitude  of 
16,000  feet.  Conflicts  occur  at  SLI  and  at  the  turn  onto  V25  with  intruders 
on  V8  and  V25,  respectively.  The  simulator  proceeds  to  Pacific  Inter¬ 
section,  where  it  receives  radar  vectors  to  V23  inbound  to  SLI.  Two  addi¬ 
tional  conflicts  occur  on  this  leg.  The  simulator  receives  vectors  for  an 
ILS  approach  to  runway  24R.  The  last  conflict  occurs  on  final  approach. 

3.4.3  Scenario  3;  Los  Angeles  to  Bakersfield 

The  simulator  departs  runway  25L  and  is  vectored  westbound  over  the 
ocean  after  being  cleared  to  FL180.  The  first  conflict  occurs  on  this  west¬ 
bound  leg.  After  the  conflict  is  resolved,  the  simulator  is  vectored  to 
intercept  V23  to  Gorman  VOR  (GMN)  and  cleared  to  its  cruise  altitude  of 
FL180.  Two  additional  conflicts  occur  before  it  reaches  GMN,  where  it  is 


vectored  to  intercept  V299  to  Fillmore  VOR  (FIM) .  One  conflict  is  generated 
on  this  leg.  After  FIM,  the  simulator  is  vectored  via  Mud  Intersection  for 
an  ILS  runway  6L  approach.  Two  conflicts  are  generated  during  this  phase, 
one  involving  a  parallel  ILS  approach. 

3.4.4  Scenario  4:  Los  Angeles  to  Las  Vegas 

The  simulator  departs  runway  7R  and  after  an  initial  climb  is  vectored 
to  intercept  V165.  The  first  conflict  occurs  when  the  simulator  crosses 
V201.  The  simulator  is  subsequently  cleared  to  its  cruise  altitude  of 
FL180  and  proceeds  to  Palmdale  VOR  (PMD)  via  V165  and  V518,  with  one  con¬ 
flict  occurring  on  each  airway.  At  PMD,  the  simulator  is  given  a  tear  drop 
turnaround  to  intercept  V197  to  Pomona  VOR  (POM) .  Two  additional  conflicts 
occur  on  V197.  At  POM,  the  simulator  is  issued  radar  vectors  to  intercept 
the  ILS  for  runway  25L.  An  intruder  is  attempting  a  simultaneous  approach 
to  runway  25R  and  causes  a  conflict. 

3.4.5  Scenario  5:  Los  Angeles  to  Santa  Barbara 

The  simulator  departs  runway  7R  and  is  vectored  south  of  the  airport 
to  intercept  V25  at  Exert  Intersection.  One  conflict  occurs  as  the  simu¬ 
lator  crosses  V201,  and  another  occurs  just  before  Exert.  After  Exert, 
the  simulator  is  cleared  to  its  cruise  altitude  of  14,000  feet.  The 
simulator  proceeds  along  V25  through  VTU  and  continues  toward  Santa 
Barbara  VOR  (SBA) .  At  this  point,  a  course  reversal  is  issued  that  takes 
the  simulator  along  V12S  to  FIM,  outbound  from  FIM  on  the  158°  radial  to 
intercept  the  ILS  approach  course  for  runway  6L.  Conflicts  occur  during 
the  turn  to  intercept  V12S,  along  V12S,  on  the  158°  radial  from  FIM,  and 
on  final  approach. 

3.4.6  Scenario  6:  Los  Angeles  to  San  Diego 

The  simulator  departs  on  runway  25L  and  is  vectored  to  intercept  V25. 

A  conflict  occurs  when  the  simulator  crosses  V201.  The  simulator  is 
cleared  to  a  cruise  altitude  of  14,000  feet  as  it  proceeds  along  V25. 

Another  conflict  occurs  on  this  leg.  A  course-reversal  is  issued  before 
the  simulator  reaches  Pacific  Intersection.  Vectors  are  issued  to  inter¬ 
cept  V23  to  SLI ,  depart  SLI  on  the  251°  radial  for  an  ILS  approach  to 
runway  6L.  Conflicts  occur  on  V23,  on  the  SLI  251°  radial,  at  the  turn 
toward  the  ILS  localizer  course,  and  at  localizer  interception. 


3. 5  TEST  PERSONNEL  AND  EXPERIMENTAL  PROCEDURES 

In  addition  to  the  flight  crew,  four  people  were  required  to  conduct 
a  simulation  session.  There  were  two  test  observers,  one  for  each  simu¬ 
lation  session,  each  having  the  same  responsibilities.  The  other  personnel 
consisted  of  a  simulation  controller  and  an  air  traffic  controller.  The 
functions  of  each  of  these  people  during  a  typical  simulation  session  are 
described  in  the  following  subsections,  and  their  normal  positions  are 
shown  in  Figure  3-1.  Experimental  procedures  are  described  in  the  context 
of  the  functional  descriptions  of  the  test  personnel. 
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3.5.1  Test  Observer 


The  test  observers  were  ARINC  Research  personnel;  they  had  several 
major  functions; 

•  Conduct  the  subject  pilot  pre-briefing 

•  Initialize  the  cockpit  simulator  for  each  scenario 

•  Coordinate  cockpit  activity  with  the  simulation  controller 

•  Observe  and  record  crew  actions  for  purposes  of  data  collection 

•  Perform  the  functions  of  the  second  officer  if  only  two  subject 
pilots  were  available 

•  Provide  voice  traffic  for  other  aircraft  in  the  scenario 

•  Conduct  the  subject  pilot  debriefing 

v 

The  pre-briefing  was  an  hour-long  briefing  of  the  ASA  concept,  experi¬ 
mental  objectives,  test  displays,  scenarios,  and  experimental  procedures. 

It  consisted  of  a  slide  show  and  videotape  presentation,  with  time  allotted 
for  questions.  The  briefing  was  conducted  in  one  of  the  training  center's 
conference  rooms.  A  copy  of  this  briefing  appears  in  Appendix  C. 

The  test  observer  then  escorted  the  subject  pilots  to  the  cockpit 
simulator  and  initialized  it  for  the  first  scenario.  This  involved  using 
the  simulator  CGI  control  panel  (Figure  2-1)  to  enter  the  planned  weather 
conditions,  aircraft  weight  and  balance  data,  fuel  and  serviceable  items, 
and  aircraft  position  data  for  the  proper  takeoff  runway.  To  minimize  the 
activity  before  takeoff,  the  test  observer  set  all  engines  running,  all 
cockpit  switches  in  the  proper  takeoff  position,  and  all  prior  systems  and 
functions  cleared,  and  positioned  the  simulator  at  the  takeoff  point, 
aligned  with  the  runway. 

Coordination  between  the  test  observer  and  the  simulation  controller 
was  accomplished  through  a  set  of  sound-powered  headsets.  After  assisting 
the  crew  with  preflight  activities  (which  included  the  duties  of  the  second 
officer  if  only  two  pilots  were  available) ,  the  test  observer  instructed 
the  simulation  controller  to  demonstrate  the  display  that  would  be  active 
in  the  upcoming  scenario.  The  test  observer  was  also  responsible  for 
informing  the  simulation  controller  of  takeoff  "rotation".  This  was  the 
cue  for  beginning  the  experimental  simulation.  Once  the  simulator  was 
airborne,  the  line  of  communication  was  used  for  discussion  of  unexplained 
phenomena  (e.g.,  equipment  problems,  performance  of  the  visual  system)  and 
general  coordination. 

During  the  flight,  the  observer  was  responsible  for  noting  the  actions 
of  the  crew  before,  during,  and  after  each  encounter.  Particular  atten¬ 
tion  was  devoted  to  recording  the  crew's  efforts  to  acquire  the  intruder 
visually  and  the  exchange  of  comments  between  crew  members.  The  test 
observer  would  answer  questions  about  a  conflict  situation  after  it  was 
resolved  but  would  not  help  the  pilots  interpret  the  ASA  displays. 
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When  the  workload  permitted,  the  test  observer  provided  voice  traffic 
on  the  ATC  frequency.  This  background  traffic  was  coordinated  with  the  air 
traffic  controller  and  represented  realistic  transmissions  for  aircraft 
flying  in  the  Los  Angeles  area.  As  a  result,  the  subject  pilots  were  often 
unable  to  talk  immediately  with  the  air  traffic  controller  during  conflict 
situations  because  of  heavy  voice  traffic. 

After  the  session  was  completed,  the  test  observer  conducted  the 
subject  pilots  back  to  the  conference  room  for  a  debriefing.  The  debriefing 
consisted  of  a  written  questionnaire  followed  by  a  discussion  of  the  experi¬ 
ment.  The  test  observer  had  the  responsibility  for  answering  questions 
about  the  questionnaire  and  leading  the  discussion  that  followed.  The 
test  observer  used  the  notes  accumulated  during  the  session  to  add  to  the 
discussion. 

Two  test  observers  were  required  to  handle  the  back-to-back  scheduling 
of  simulator  time.  A  typical  schedule  appears  in  Figure  3-2. 


Crew  B  Crew  B 

Pre-Briefing  Simulator  Section 


Crew  B 
Debriefing 


Crew  A  1  Crew  A  1  Crew  A  1 

Pre-Briefing  Simulator  Session  Debriefing 

I _ I _ I _ I _ I _ I _ I _ I 

1900  2000  2100  2200  2300  2400  0100  0200 

Figure  3-2.  TYPICAL  CREW  SCHEDULING 


When  time  permitted,  the  second  test  observer  assisted  in  providing  back¬ 
ground  voice  traffic. 

3.5.2  Simulation  Controller 


The  simulation  controller,  an  ARINC  Research  employee,  had  the  follow¬ 
ing  functions : 

•  Experimental  setup  and  checkout 

•  Interaction  with  the  simulation  software 

•  Coordination  with  the  air  traffic  controller 

•  Management  of  data  recording  equipment  and  material 

Before  each  simulation  session,  the  simulation  controller  was  respon¬ 
sible  for  setting  up  and  checking  out  the  simulation  control  computer,  the 
CGI  computer,  and  the  communications  links  between  the  simulator  and  the 
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control/monitor  station.  As  soon  as  the  simulator  was  available,  the 
simulation  controller  coordinated  with  United  Airlines  maintenance  person¬ 
nel  for  installation,  power  up,  and  checkout  of  the  ASA  displays.  At 
"rotation,"  the  simulation  controller  initiated  the  ASA  simulation. 

The  simulation  controller  was  the  simulation's  computer  operator  and 
conducted  all  of  the  interaction  with  the  computers.  The  interactive 
functions  included  the  following: 

•  Obtaining  information  about  aircraft  active  in  the  scenario 

•  Releasing  and  controlling  intruder  aircraft 

•  Rescaling  the  ATC  display 

•  Changing  the  ASA  mode  of  operation 

To  augment  the  information  presented  on  the  air  traffic  controller's 
display,  the  simulation  controller  could  request  additional  information 
on  aircraft  adtive  in  the  scenario  (including  the  simulator) .  This  infor¬ 
mation  includes  position  coordinates  (x,  y,  z) ,  heading,  and  indicated 
airspeed  (IAS) . 

The  intruder  geometry  and  nominal  conflict  location  were  preprogrammed, 
but  actual  release  and  control  of  the  intruder  was  the  function  of  the 
simulation  controller.  The  nominal  release  point  was  based  on  thee, simu¬ 
lator’s  being  a  given  distance  from  a  given  point.  However,  if  the  simu¬ 
lator  was  not  at  the  expected  altitude  or  was  off  the  nominal  flight  path, 
the  encounter  could  be  delayed  by  the  simulation  controller  without  affect¬ 
ing  the  desired  conflict  geometry.  In  addition,  if  after  release  it  appeared 
that  the  conflict  would  not  occur  (exclusive  of  any  evasive  actons  taken 
by  the  pilot),  the  simulation  controller  could  modify  the  intruder's 
altitude  and  speed. 

\ 

Coordination  with  the  air  traffic  controller  included  the  functions 
mentioned  above  as  well  as  keeping  the  controller  abreast  of  upcoming  > 
events  in  the  scenario  and  relaying  messages  from  the  test  observer.  Vv- 

Data  recording  was  for  the  most  part  automatic.  The  simulation  con¬ 
troller  was  responsible  for  mounting  tapes,  transferring  data  from  disk  to 
tape,  and  documenting  the  recorded  data. 

3.5.3  Air  Traffic  Controller 

The  air  traffic  controller  who  participated  in  the  experiment  was 
Chester  MacMillan,  an  8-year  veteran  of  the  FAA.  Mr.  MacMillan  is  a  Plans 
and  Procedures  Specialist  with  the  Rocky  Mountain  Region  and  was  involved 
in  the  experimental  metering  and  spacing  and  profile  descent  programs 
operated  at  Denver. 

The  air  traffic  controller's  primary  function  was  to  provide  control 
for  the  cockpit  simulator  and  provide  background  voice  traffic  for  other 
simulated  aircraft.  The  controller  simulated  all  of  the  ATC  positions  — 
clearance  delivery,  tower,  departure,  en  route,  and  approach  —  and 
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simulated  handoffs  between  the  positions.  The  subject  pilots  received  all 
of  the  clearances  that  they  would  normally  receive  in  a  typical  flight 
around  Los  Angeles.  The  nominal  clearances  given  for  each  of  the  six 
scenarios  appear  in  Appendix  A. 

The  controller  was  also  responsible  for  ensuring  that  the  conflict 
situations  developed  as  planned.  This  would  sometimes  require  the  con¬ 
troller  to  issue  special  clearances  to  maneuver  the  simulator  into  position. 

The  controller  was  assisted  by  the  test  observer  and  was  presented  with 
a  plan  view  of  the  traffic  situation  (Figure  3-3) .  This  graphical  pre¬ 
sentation  was  displayed  on  a  storage  tube  device  and  included  appropriate 
airways,  fixes,  aircraft  tracks,  and  a  readout  of  simulator  altitude  (in 
hundreds  of  feet) .  Aircraft  were  represented  by  letters  to  distinguish 
the  tracks.  The  simulator  was  always  aircraft  "K".  The  display  would 
automatically  redraw  and  center  the  simulator  when  its  path  approached  the 
edge  of  the  viewing  area.  This  rescaling  could  also  be  accomplished  on 
request. 


Figure  3-3.  AIR  TRAFFIC  CONTROLLER  DISPLAY 
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CHAPTER  FOUR 


QUALITATIVE  DATA  ANALYSIS 


The  subjective  evaluations  of  the  pilot  population  can  be  as  decisive 
as  the  technical  measurements  of  the  system  in  determining  the  acceptability 
of  the  ASA  concepts.  Therefore,  a  subjective,  or  qualitative,  analysis  was 
performed  as  part  of  the  simulation  evaluation.  The  data  collected  were 
in  the  form  of  questionnaires  and  debriefing  comments. 


4.1  DATA  COLLECTION  PROCEDURES 

Immediately  following  the  simulator  test  session,  the  subject  pilots 
were  asked  to  fill  out  a  lengthy  questionnaire  that  included  questions  on 
pilot  background,  quality  of  displayed  commands,  display  characteristics, 
display  usability,  traffic  advisories,  command  presentation,  ASA  concepts, 
display  evaluation,  and  the  ASA  program. 

After  completing  the  questionnaire,  the  pilots  were  subjected  to  a 
debriefing  session  in  which  several  issues  were  probed  more  deeply,  and  the 
pilots  were  asked  to  provide  additional  comments  on  any  aspect  of  the 
simulation  or  ASA  concept  that  they  desired.  The  discussion  was  led  by 
the  test  observer,  who  used  his  observations  during  the  simulation  to 
stimulate  the  conversation.  Issues  that  were  discussed  in  all  of  the 
sessions  included  preference  for  horizontal  versus  vertical  maneuvers, 
additional  flight  testing,  realism  of  the  visual  scene,  use  of  the  color 
red  for  collision  avoidance  commands,  comparisons  of  active  versus  full 
mode,  relative  versus  absolute  data  presentations,  value  of  the  vertical 
speed  limit  commands,  and  the  role  of  the  ASA  system  in  the  operational 
environment. 

The  original  questionnaire  data  were  supplemented  by  a  follow-up 
questionnaire  approximately  2  1/2  months  after  the  close  of  testing.  The 
second  questionnaire  was  developed  to  help  clarify  some  of  the  questions 
that  were  previously  asked.  In  addition,  pilots  were  solicited  for  addi- 
ional  comments  and  changed  opinions.  The  questionnaires  are  reproduced 
.n  Appendixes  D  and  E. 
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4.2  QUESTIONNAIRE  ORGANIZATION 


The  initial  questionnaire  consisted  of  53  questions  distributed 
among  the  following  subject  areas: 

•  Subject  pilot  background 

•  Test  display  evaluation 

•  Information  content  of  an  ASA  display 

•  Impact  of  introduction  of  ASA  on  flight  and  ATC  procedures 

All  74  subject  pilots  filled  out  a  questionnaire.  However,  the  first 
three  crews  (10  subject  pilots)  filled  out  a  shorter  version.  Discussions 
with  the  FAA  led  to  the  more  complete  version  given  to  the  remaining  crews. 

The  type  of  questions  asked  in  the  questionnaire  included  multiple 
choice,  yes  and  no,  single  answer,  and  brief  comments.  In  addition, 
comments  were  solicited  on  any  of  the  questions  to  which  the  pilots  wished 
to  respond.  In  explaining  the  results,  this  report  employs  some  of  those 
comments . 

The  supplemental  questionnaire  has  a  format  similar  to  that  of  the 
initial  questionnaire.  The  questions  were  designed  to  clarify  previously 
asked  questions  and  to  verify  that  the  questions  were  properly  understood. 
Several  questions  were  included  to  determine  if  pilot  opinion  had  changed 
in  the  2-1/2  months  since  the  close  of  testing.  The  supplemental  question¬ 
naire  was  mailed  to  all  74  subject  pilots,  and  50  responded. 


4.3  ANALYSIS  METHODOLOGY 

The  analysis  of  the  questionnaire  data  involved  tabulation  of  the 
responses.  Since  the  underlying  distributions  of  the  data  are  unknown 
because  of  the  nonquantif iable  nature  of  the  responses,  a  nonparametric 
(i.e.,  distribution-free)  statistical  analysis  approach  was  applied  to 
evaluate  the  data.  The  nonparametric  methods  are  based  on  the  relative 
ranking  of  the  responses  (least  to  greatest)  as  a  replacement  for  actual 
numerical  measurements  of  the  type  used  in  classical  statistical  methods. 

The  following  factors  were  considered  in  the  analysis  as  having 
influence  on  the  results: 

•  Position  (captain,  first  officer,  and  second  officer) 

•  Currency  (currently  B-727-rated) 

•  Familiarity  with  United  B-727  simulator 

•  Familiarity  with  Los  Angeles  airspace 

•  Subject  pilot  selection  (recommended  by  airline  management  or 
Air  Line  Pilots  Association) 

•  Display  preference 
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Position  is  related  to  the  crew's  experience  since  airline  captains 
usually  have  more  flying  hours  than  first  or  second  officers.  Familiarity 
with  the  B-727,  the  United  simulator,  and  the  Los  Angeles  airspace  have  an 
impact  on  the  pilot's  ability  to  fly  the  simulator  and  therefore  may  affect 
his  performance  in  a  conflict  situation.  Subject  pilot  selection  is 
treated  as  a  factor  to  ensure  that  the  results  were  not  overly  influenced 
by  either  airline  management's  or  ALPA's  point  of  view.  Display  preference 
is  used  as  a  factor  to  eliminate  any  preconceived  bias  toward  any  of  the 
displays  used. 


4 . 4  QUALITATIVE  RESULTS 

The  results  are  illustrated  in  bar  chart  form.  Each  chart  represents 
a  particular  topic  and  presents  the  questions  and  possible  responses 
exactly  as  they  appeared  in  the  subject  questionnaires.  The  results  are 
presented  as  a  percentage  of  those  pilots  responding  to  the  question.  The 
actual  number  of  pilots  responding  follows  in  parentheses.  Since  some 
questions  were  left  unanswered,  comparisons  between  responses  should  be 
made  by  comparing  percentage  responding  as  opposed  to  number  responding. 

4.4.1  Test  Display  Evaluation 

This  section  addresses  test  display  physical  characteristics  such  as 
readability  and  under standability ,  use  of  color,  displayed  range,  alerting 
features,  and  display  format.  It  also  examines  test  display  preference 
and  suggested  alternatives.  While  the  questions  asked  were  related 
primarily  to  the  test  displays,  conclusions  can  be  drawn  about  features 
that  would  be  desirable  or  undesirable  in  a  generic  display. 

4. 4. 1.1  Readability  and  Understandability 

The  questions  on  readability  served  the  purpose  of  substantiating 
that  the  selected  test  displays  were  of  an  acceptable  test  quality  and 
that  the  presentation  was  understood.  This  was  confirmed  by  the  subject 
pilots.  More  than  80  percent  of  the  pilots  felt  that  the  three  displays 
were  usually  or  always  readable  and  understandable.  These  results  were 
generally  uniform  over  the  possible  commands,  but  there  were  some  problems 
with  presentation  of  "No  Turn"  and  "Limit  Climb"  commands  on  the  IVSI. 

The  limit-climb  bars  were  obscured  somewhat  by  the  top  of  the  instrument 
case  because  of  the  angle  at  which  the  instrument  was  viewed,  and  the 
placement  of  the  lights  for  the  "No  Turn"  indicators  was  insufficient  for 
total  illumination.  Other  comments  reflected  the  difference  in  intensity 
between  the  two  LED  displays.  These  two  prototype  instruments  were 
identical  in  format;  however,  one  instrument  was  designed  for  both  daylight 
and  night  applications  and  the  other  for  low  ambient  light  applications. 

On  the  CRT  display,  the  problems  concerned  the  obstruction  of  the  lower 
left  portion  of  the  CRT  by  the  speed  brake  lever;  this  would  block  parts 
of  the  negative  and  limit  commands  ("Don't"  and  "Limit")  when  viewed  from 
the  captain's  position. 
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4. 4. 1.2  Use  of  Color 


The  IVSI  used  two  colors  in  its  presentation  —  red  for  positive 
commands  and  yellow  for  negative  and  limit  vertical  commands.  The  LED 
used  those  colors  and  added  green  for  traffic  advisories.  The  CRT  display 
was  monochromatic. 

The  subject  pilots  were  asked  to  give  their  reactions  to  the  use  of 
color  in  the  IVSI  and  LED  and  to  the  proposal  for  using  color  in  the  CRT. 
The  responses  favored  its  use,  as  illustrated  in  Figure  4-1.  In  addition, 
the  use  of  color  was  suggested  as  an  alternative  to  "filling  in"  the 
intruder  symbol  on  the  CRT  as  a  means  of  differentiating  between  the  PWIs 
and  actual  threats. 


Questions 

Answers 

Did  the  use  of  color 

very  much 

(green  for  advisories, 

yellow  for  maneuver  limits, 

and  red  for  commands) 

some 

help  in  interpreting 

the  information  presented 

on  the: 

very  little 

IVSI 

LED  mm  mm 

none 

Would  the  use  of  colors 

very  much 

(green  for  advisories. 

yellow  for  maneuver  limits, 

some 

and  red  for  commands) 

help  you  in  interpreting 

very  little 

the  information  presented 

on  the  CRT? 

none 

Percent  Responding 
50  75 


Figure  4-1.  USE  OF  COLOR 


The  issue  of  using  red  in  an  ASA  display  to  indicate  a  positive  action 
was  focused  on  in  the  debriefing.  The  vast  majority  of  crews  felt  that  red 
was  used  appropriately  in  the  experimental  displays,  a  few  felt  that  it 
was  an  important  feature,  and  others  objected  slightly  but  felt  that  proper 
training  would  eliminate  any  problems.  Very  few  voiced  strong  objections. 


Those  who  objected  cited  the  traditional  connotation  of  red  in  a  display 
to  stop  a  particular  action  and  not  to  indicate  a  positive  action. 

4. 4. 1.3  Displayed  Range 

Only  the  LED  and  CRT  displays  presented  range  information  on  other 
aircraft.  The  LED  displayed  range  as  part  of  the  traffic  advisory  message 
and  presented  it  as  a  two-digit  integer  representing  the  number  of  nautical 
miles  from  the  simulator.  The  CRT  had  two  presentations  for  range.  In 
full  mode,  range  rings  divided  the  display  area  at  3-nautical  mile  and 
6-nautical  mile  intervals  from  the  simulator,  with  a  maximum  displayed 
range  of  9  miles  ahead  and  6  miles  behind  and  on  both  sides.  In  active 
mode,  range  was  presented  as  a  number  accurate  to  tenths  of  a  mile. 

Responses  to  the  questionnaire  indicated  that  the  pilots  were  for  the 
most  part  satisfied  with  the  range,  as  illustrated  in  Figure  4-2.  Comments 
suggested  a  multiple  scale  on  the  CRT  that  could  be  adjusted  in  flight. 


Questions 

Answers 

Percent  Responding 

25  50  75  100 

Was  the  range  information 
used  for  this  test 
satisfactory? 

LED  —  m 

CRT  immi 

very  much 

some 

little 

none 

--(6) 

""  (5) 

(0) 

” _ i _ i _ i _ i _ 

Figure  4-2.  DISPLAYABLE  RANGE 


4. 4. 1.4  Alerting  Features 

The  primary  alerting  feature  for  all  three  of  the  tested  displays  was 
a  2-second  tone  burst  (1  kHz  square  wave) .  The  alarm  was  sounded  whenever 
a  new  limit,  negative,  or  positive  command  was  presented  on  the  display. 

Two  questions  were  asked  about  alerting  features,  the  results  of  which 
are  presented  in  Figure  4-3.  A  general  comment  indicated  that  the  alarm 
could  have  been  louder  (one  pilot  said  that  he  never  heard  the  alarm) . 

Comments  from  those  that  answered  negatively  on  the  necessity  of  an 
audio  alert  restricted  their  answer  to  a  nighttime  scenario.  Many  felt 
that  during  the  daylight  hours  the  alert  would  be  more  important.  Display 
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Questions 


Do  you  feel  that  the 
audio  alert  was  necessary 
to  draw  your  attention  to 
commands  on  the: 

IVSI  ii  ■ 

LED  —  -■ 

CRT 


Would  a  voice  command 
be  desirable  for  the  ASA 
function  (e.g.,  "Turn  Right 
Whoop  Whoop  Turn  Right")? 


Answers 


Percent  Responding 
25  50  75 


always 

usually 

about  half 
the  time 

sometimes 

never 


i<24) 

—  127) 

•■•■■■(29) 


_i  (18) 

1  (17) 
'••••(25) 


(18) 

•  •(25) 

klHMI  (13) 


'(15) 

'(15) 


■  (7) 
„  '(6) 
■■■'(7) 


very 

desirable 


acceptable 

undesirable 


100 


Figure  4-3.  ALERTING  FEATURES 


location  also  had  some  impact  on  the  results,  although  not  very  signifi¬ 
cant.  Those  pilots  who  answered  differently  for  the  three  displays  felt 
that  location  was  a  factor  and  that  the  alarm  was  required  more  for  the 
CRT  since  it  was  located  away  from  the  main  instrument  cluster. 

The  responses  to  the  voice  command  option  may  have  been  biased 
because  of  the  unfortunate  use  of  the  phrase  "whoop  whoop"  in  the  example 
Many  pilots  instantly  related  the  voice  command  to  the  ground  proximity 
warning  system  installed  aboard  all  aircraft.  This  device,  as  initially 
installed,  produced  a  large  number  of  false  alarms  and  therefore  was  not 
well  received  by  pilots. 

Other  comments  related  to  the  negative  aspects  of  startling  alarms 
are  as  follows : 

"Startle  factor  temporarily  prevents  any  action." 

"The  more  annoying,  the  longer  it  takes  to  recover  one's  thought 

process . ” 

"Crew  reaction  is  to  disable  warring  rather  than  solve  the  problem 

(Fire/GPWS  Warning  Bell)." 
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"Even  very  loud/annoying  alerts  can  be  mentally  tuned  out  or  disregarded/ 

ignored. " 

A  secondary  alerting  factor  is  related  to  the  visual  cues  produced  by 
a  newly  displayed  command.  This  feature  was  accentuated  by  the  nighttime 
simulation  and  benefited  the  IVSI  most,  the  LED  somewhat,  and  the  CRT  least. 
Location  of  the  display  is  the  dominant  factor,  although  intensity  and 
contrast  levels  have  a  significant  effect. 

4.4.1. 5  Display  Format 

This  section  presents  display-format  comments  not  discussed  in  the 
previous  sections. 

A  specific  format  item  investigated  was  the  presentation  of  positive 
commands  on  an  ASA  display.  The  pilots  had  been  presented  with  arrows 
only  (IVSI)*,  arrows  and  text  (LED),  and  text  only  (CRT).  The  results 
showed  that  25  percent  of  the  subject  pilots  preferred  text  only,  41  percent 
preferred  arrows  only,  and  the  remaining  34  percent  preferred  a  combination 
of  text  and  arrows. 

Other  comments  were  specific  to  the  formats  actually  used  in  the  experi¬ 
ment.  Several  comments  illustrated  concern  over  the  LED  presentation  of 
traffic  advisories.  Many  felt  that  the  format  required  too  much  inter¬ 
pretation  and  that  the  display  was  too  busy  when  multiple  traffic  advisories 
were  presented.  There  were  also  some  problems  with  interpreting  the 
traffic  advisories.  Pilots  would  confuse  the  order  in  which  the  informa¬ 
tion  was  presented  and  forget  the  units  in  which  the  information  was 
measured. 

The  most  common  complaint  about  the  CRT  display  was  the  small  size  of 
the  symbology  used  (0.10"  x  0.14")  for  the  aircraft  alphanumeric  tags.  The 
size  was  a  limitation  of  the  electronics  and  was  related  to  the  available 
dot  size.  Other  comments  suggested  moving  the  alphanumeric  tag  away  from 
the  aircraft  symbol  and  trail  for  improved  readability. 

Many  pilots  commented  on  the  active  mode  format  for  the  CRT.  They 
felt  that  there  was  too  much  wording  on  the  display  and  that  no  wording 
was  required  in  nonthreatening  situations. 

4.4. 1.6  Test  Display  Preference 


This  section  examines  the  subject  pilots'  preferences  for  the  test 
displays  used  in  the  experiment.  Since  the  field  of  test  displays  included 
only  three  representatives,  the  results  presented  in  this  section  should 
be  interpreted  as  a  preference  for  a  display  presentation  (lighted  indica¬ 
tors,  alphanumerics ,  and  pictorial  display  with  alphanumerics)  or  for 
information  content  (commands  only,  commands  with  alphanumeric  traffic 


*The  IVSI  is  considered  an  arrows-only  presentation  because  the  lettering 
on  the  arrows  is  judged  to  be  too  small  to  interpret  quickly. 


4-7 


advisories,  and  commands  with  a  pictorial  traffic  situation).  The  results 
should  not  be  interpreted  as  a  recommendation  for  implementing  one  of  the 
test  displays. 

The  subject  pilots  were  asked  to  rate  the  displays  in  order  of  prefer¬ 
ence  based  on  the  simulation  session.  The  results  are  shown  in  Figure  4-4. 
While  the  results  are  not  overwhelmingly  in  favor  of  any  particular  display, 
there  is  a  definite  preference  for  the  CRT  display.  The  LED  display  was 
ranked  second  and  the  IVSI  third  in  overall  preference.  However,  as  illus¬ 
trated  in  Figure  4-5,  any  of  the  three  displays  would  be  acceptable  to  a 
large  percentage  of  the  sampled  pilot  population  (more  than  73  percent) . 
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Figure  4-4.  TEST  DISPLAY  PREFERENCE  (OVERALL) 

(ORIGINAL  QUESTIONNAIRE) 

The  data  were  categorized  to  determine  the  effect  of  the  sponsoring 
organization  on  the  results.  The  two  categories  are  pilots  recommended 
by  the  Air  Line  Pilots  Association  (ALPA)  and  pilots  recommended  by  airline 
management.  The  results  are  presented  in  Figures  4-6  and  4-7.  The 
management-recommended  pilots  were  somewhat  evenly  distributed  with  regard 
to  display  preference  although  the  ranking  remained  the  same  as  in  the 
overall  breakdown.  The  ALPA- re commended  pilots  also  indicated  the  same 
ranking. 

A  further  breakdown  of  display  preference  by  pilot  experience 
(captains  versus  first  officers)  was  also  performed.  Again,  the  results 
(Figures  4-8  and  4-9)  indicated  basically  the  same  preferences. 

Similar  disp) ay-preference  questions  were  asked  in  the  supplemental 
questionnaire;  the  results  are  shown  in  Figures  4-10  and  4-11.  The 
questions  were  restricted  to  the  active  mode  environment,  where  bearing 
information  was  not  available.  The  questions  also  suggested  that  the 


First  Choice  Second  Choice  Third  Choice 


IVSI  LED  CRT  IVSI  LED  CRT  IVSI  LED  CRT 


Figure  4-5.  ACCEPTABILITY  OF  SECOND  AND  THIRD  CHOICES 
(ORIGINAL  QUESTIONNAIRE) 
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Figure  4-6.  TEST  DISPLAY  PREFERENCE  (ALPA  -  RECOMMENDED  PILOTS) 
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Figure  4-7.  TEST  DISPLAY  PREFERENCE  (MANAGEMENT  -  RECOMMENDED  PILOTS) 


First  Choice  Second  Choice  Third  Choice 
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First  Choice  Second  Choice  Third  Choice 


Figure  4-10.  TEST  DISPLAY  PREFERENCE 
(OVERALL) (SUPPLEMENTAL 
QUESTIONNAIRE) 


Questions 

Answers 

Percent  Responding 

25  50  75  100 

Would  your  second 
choice  be  acceptable 
if  the  first  choice 
was  not  available? 

Would  your  third 
choice  be  acceptable 
if  the  first  and 
second  choices  were  not 
available? 

yes 

no 

yes 

no 

till 

-<3) 

_ 1 _ 1 _ 1 _ 1 - 

Figure  4-11.  ACCEPTABILITY  OF  SECOND  AND  THIRD  CHOICES 
(SUPPLEMENTAL  QUESTIONNAIRE) 


active  mode  representation  on  the  CRT  display  would  most  probably  be  a 
graphical  presentation  rather  than  the  alphanumeric  presentation  used 
in  the  tests. 

The  results  still  maintain  the  same  overall  ranking  of  the  displays, 
with  the  CRT  and  LED  displays  losing  some  ground  to  the  IVSI,  making  the 
results  appear  more  uniformly  distributed.  In  addition,  there  appeared 
to  be  much  less  acceptance  of  the  pilots'  third  choice. 
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4.4.1. 7  Display  Alternatives 


This  section  examines  possible  improvements  to  the  test  displays, 
display  combinations,  and  alternative  instrumentation. 

The  IVSI  was  the  only  display  device  without  traffic  advisory 
information.  Therefore,  the  response  from  several  pilots  to  add  range 
and  bearing  to  conflicting  aircraft  was  expected.  In  addition,  the  IVSI 
was  the  only  device  that  combined  the  ASA  function  with  another  display 
function,  and  there  was  concern  that  modification  of  the  IVSI  by  the 
addition  of  command  lights  would  detract  from  the  primary  purpose  of  the 
instrument.  More  than  90  percent  of  the  pilots  felt  that  there  was  no 
problem. 

Suggested  improvements  for  the  LED  display  reflected  the  cluttered 
appearance  that  the  display  presented  to  some  of  the  subject  pilots. 
Comments  suggested  eliminating  multiple  traffic  advisories,  only  showing 
the  most  important;  and  eliminating  direction  of  flight  from  the  advisory. 

Combinations  of  displays  were  also  suggested.  More  than  50  percent 
of  the  pilots  responding  felt  that  a  combination  of  display  devices  would 
provide  more  usable  information  than  the  best  test  display.  The  IVSI 
and  CRT  combination  was  best  received  by  26  percent  of  the  pilots,  and  the 
IVSI  and  LED  combination  and  LED  and  CRT  combination  each  received  15  per¬ 
cent  of  the  pilots’  votes. 

A  third  of  the  subject  pilots  also  recommended  alternative  aircraft 
instruments  that  could  be  modified  to  provide  ASA  information.  These 
devices  are  listed  below  in  order  of  preference  (the  number  of  pilots 
suggesting  these  displays  is  included  in  parentheses) : 

•  Horizontal  Situation  Indicator  (HSI) /Electronic  Horizontal 
Situation  Indicator  (EHSI)  (7) 

•  Artificial  Horizon/Flight  Director  (5) 

•  Weather  Radar  (3) 

•  Altimeter  (2) 

•  Vertical  Speed  Indicator  (VSI) /Horizontal  Situation  Indicator 
(HSI)  combination  (2) 

•  Head-Up  Display  (HUD)  (2) 

•  Radio  Magnetic  Indicator  (RMI)  (1) 

4.4.2  Information  Content  of  an  ASA  Display 


This  section  addresses  information  elements  that  could  be  part  of 
an  ASA  display.  In  addition,  it  examines  the  combination  of  those  elements 
into  traffic  advisories  and  commands. 


4.4. 2.1  Information  Elements 

In  the  questionnaire,  the  subject  pilots  were  presented  with  11  collision 
avoidance  information  elements  that  they  were  asked  to  rank  (1  to  11)  in 
order  of  importance.  They  were  also  asked  to  check  all  elements  that  they 
considered  to  be  an  essential  part  of  any  collision  avoidance  display. 

The  results  were  tabulated  and  a  composite  rank  for  each  element  was 
produced.  The  composite  rank  was  calculated  as  the  sum  of  the  rank  values 
(1  to  11)  selected  by  the  pilots  divided  by  the  number  of  pilots  responding. 
The  elements  are  listed  in  Table  4-1  in  order  of  rank  (the  value  1  indicates 
the  highest  rank)  with  their  composite  rank  and  the  percentage  of  respondents 
that  considered  the  information  item  essential  to  a  collision  avoidance 
display. 


Table  4-1.  RANKING  OF  INFORMATION  ITEMS 

Rank 

Information  Item 

Composite 

Rank 

Percent 

Responding 

"Essential" 

Altitude  of  other  aircraft 

1.787 

85 

Range  of  other  aircraft 

2.686 

81 

Relative  bearing 

3.542 

59 

Heading  of  other  aircraft 

4.089 

51 

5 

Horizontal  closure  rate 

4.801 

21 

6 

Vertical  closure  rate 

5.524 

17 

7 

Vertical  speed  of  other  aircraft 

6.713 

20 

8 

Projected  miss  distance 

7.421 

14 

Closure  angle 

7.425 

7 

Other  aircraft  type 

9.198 

5 

D 

Other  aircraft  identity 

10.052 

3 

These  results  show  that  altitude  and  range  of  the  other  aircraft  are  by 
far  the  most  important  information  elements  for  a  collision  avoidance 
display.  This  information  is  available  directly  from  the  active  mode  ASA 
data  base.  Bearing  and  heading  of  other  aircraft,  third  and  fourth 
ranking,  were  considered  essential  by  more  than  half  of  the  subject  pilots. 
This  information  is  available  only  if  the  full  mode  ASA  system  is  in  use. 
The  remaining  elements  were  rated  significantly  lower  and  were  considered 
much  less  critical  to  conflict  resolution. 
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With  the  exception  of  altitude,  all  of  the  information  elements  could 
be  clearly  expressed  in  either  relative  or  absolute  terms,  but  not  both. 

To  determine  the  preferred  representation  of  altitude  information,  a  question 
was  included  in  the  questionnaire.  The  overwhelming  majority  (89  percent) 
favored  an  MSL  or  absolute  representation  versus  a  relative  representation. 

4.4. 2. 2  Traffic  Advisories 

Traffic  advisories  were  presented  on  both  the  LED  and  CRT  displays. 

The  advisories  were  initiated  by  the  ASA  PWI  logic.  The  logic  defined  a 
protected  volume  of  airspace  around  an  aircraft  and  identified  those 
aircraft  which  violated  it.  The  protected  volume  was  defined  with  range, 
range  rate,  horizontal  tau,  altitude,  altitude  rate,  and  vertical  tau 
criteria.  Once  displayed,  the  advisory  remained  visible  for  the  amount  of 
time  the  aircraft  was  within  the  protected  volume;  however,  the  display 
logic  maintained  the  advisory  for  at  least  30  seconds.  This  30-second 
period  was  selected  as  a  result  of  pre-test  design  evaluation.  Advisories 
appearing  for  less  than  that  time  were  considered  distracting.  Advisories 
were  updated  every  second  in  active  mode  and  every  4  seconds  in  full  mode. 

Up  to  three  advisories  could  be  displayed  concurrently  on  the  LED  in 
nonconflict  situations.  When  an  ASA  command  was  required,  however,  only 
the  advisory  for  the  aircraft  involved  in  the  conflict  was  displayed.  The 
format  for  the  LED  traffic  advisories  is  specified  in  Subsection  2.3.2. 

The  CRT  display  was  capable  of  displaying  all  aircraft  that  violated 
the  PWI  criteria  and  fell  within  the  range  limitations  of  the  display  (9 
miles  forward  and  6  miles  behind  and  to  both  sides  of  own-aircraft) . 
Typically,  in  the  traffic  environment  used  in  the  simulation,  no  more  than 
three  advisories  were  presented.  However,  occasionally  more  were  presented. 
When  an  ASA  command  was  presented,  the  symbol  representing  the  aircraft 
involved  in  the  conflict  became  solid.  All  other  advisories  remained  dis¬ 
played.  The  format  for  the  CRT  traffic  advisories  is  described  in  Subsec¬ 
tion  2.3.3. 

There  were  a  number  of  questions  in  the  initial  and  supplemental 
questionnaires  regarding  traffic  advisories.  The  results  are  presented 
in  Figures  4-12  through  4-14. 

Figure  4-12  illustrates  the  results  of  questions  on  the  information 
content  of  traffic  advisories.  Most  pilots  felt  that  the  advisories  were 
presented  in  a  useful  format  on  both  the  LED  and  CRT  displays,  with  many 
pilots  feeling  that  more  aircraft  than  necessary  were  sometimes  displayed 
on  the  two  devices.  This  was  verified  by  a  separate  question  requesting 
a  recommended  maximum  number  of  simultaneously  displayed  advisories. 

Figure  4-13  illustrates  the  results.  The  average  recommendation  for  the 
LED-type  format  was  slightly  less  than  2.5  advisories,  with  a  CRT-type 
recommendation  of  slightly  greater  than  3  advisories.  This  suggests  that 
a  pictorial  presentation  is  simpler  to  understand;  however,  the  CRT  display 
area  was  several  times  larger  than  the  display  area  for  the  LED,  which 
permitted  a  less  cluttered  display.  These  data  also  reflect  the  pilot's 
inability  to  handle  more  than  2  or  3  simultaneous  advisories. 


i 


Questions 


Were  traffic  advisories 
presented  in  a  useful 
format  on  the  .  .  . 


Answers 


Percent  Responding 
25  50  75 


Were  more  aircraft 
advisories  than  necessary 
displayed  on  the  .  .  . 


During  this  simulation, 
were  you  able  to 
visually  acquire  traffic 
and  quickly  correlate  it 
with  the  advisories 
presented  on  the  .  .  . 


Figure  4-12.  TRAFFIC  ADVISORIES  (INFORMATION  CONTENT) 
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Figure  4-13.  RECOMMENDED  NUMBER  OF  SIMULTANEOUSLY 
DISPLAYED  AIRCRAFT  ADVISORIES 


Other  results  indicate  that  the  displays  were  useful  in  visual  acqui¬ 
sition  of  traffic.  The  pilots  were  able  to  acquire  traffic  and  quickly 
correlate  it  to  the  displayed  advisories  as  well  as  locate  traffic  that 
they  would  not  ordinarily  see. 

A  final  question  on  advisory  information  content  asked  if  the  advisory 
information  was  sufficient  to  minimize  deviations  from  the  planned  flight 
path.  Many  pilots  (75  percent)  felt  that  that  was  always  or  often  the  case. 

A  smaller  percentage  believed  that  it  was  true  at  least  some  of  the  time.  The 
important  result  is  indicated  by  the  preference  for  the  pictorial  presenta¬ 
tion  over  an  alphanumeric  message  with  regard  to  minimizing  flight  path 
deviations . 

Figure  4-14  examines  the  utility  of  the  traffic  advisories  in  an  air¬ 
craft  separation  assurance  system.  For  the  most  part,  pilots  felt  that  the 
advisories  were  presented  in  time  to  be  useful  and  that  they  were  about  as 
useful  or  more  useful  than  verbal  advisories  from  ATC.  Ninety  percent  of 
the  pilots  felt  that  traffic  advisories  were  an  essential  part  of  ASA. 

Since  there  was  possible  confusion  between  displayed  and  verbal  advisories 
with  regard  to  this  question,  the  supplemental  questionnaire  asked  if  the 
advisories  were  essential  for  an  ASA  display.  The  results  were  still  over¬ 
whelmingly  positive  (79  percent)  but  less  than  for  the  original  question. 
Finally,  the  supplemental  questionnaire  investigated  active  mode  advisories, 
and  the  results  indicated  the  same  positive  attitude. 

Discussions  with  the  pilots  provided  insight  into  the  problems  asso¬ 
ciated  with  today's  method  of  issuing  traffic  advisories.  Pilots  complained 
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I 


Questions 

Answers 

Were  traffic  advisories 
presented  in  time  to  be 
useful? 

always 

often 

sometimes 

never 

Were  the  ASA  traffic 
advisories  as  useful 
as  verbal  advisories 
from  ATC? 

more 

useful 

about  as 
useful 

LED  —  — 

CRT  ••«*••• 

seldom  as 
useful 

never  as 
useful 

Do  you  feel  some  form 
of  traffic  advisories 
are  an  essential  part 
of  ASA? 

$  o 
>  c 

_ 

Do  you  feel  that 
traffic  advisories  are 
an  essential  part  of 
a  BCAS  display? 

$  o 
>  c 

_ 

If  the  traffic 
advisories  contain  only 
altitude  and  range 
of  the  surrounding 
traffic  (a  limitation 
of  active  BCAS),  would 
they  still  be  considered 
an  essential  part 
of  a  BCAS  display? 

yes 

no 

Percent  Responding 
25  50  75 


Figure  4-14.  TRAFFIC  ADVISORIES  (UTILITY) 


4-17 


that  too  many  aircraft  are  reported  and  that  only  a  small  percentage  are 
actually  acquired  visually.  Once  acquired,  many  are  lost  or  identified 
incorrectly.  In  addition,  it  is  sometimes  difficult  for  the  pilot  to 
recognize  when  traffic  is  no. longer  threatening.  Another  problem  is 
related  to  high-density  situations,  when  advisories  are  most  important 
but  the  controller  is  unable  to  issue  them  because  of  a  heavy  workload. 

The  displayed  advisories  complement  the  present  system  and  solve  many  of 
these  problems. 

4. 4. 2. 3  ASA  Commands 

Commands  were  presented  on  all  three  of  the  test  displays.  The  IVSI 
presented  lighted  indicators;  the  LED  displayed  alphanumerics  and  arrows; 
and  the  CRT  displayed  alphanumerics  only..  The  logic  to  produce  the  commands 
was  the  same  for  all  three  devices.  The  commands  consisted  of  positive 
horizontal  (TURN  RIGHT,  TURN  LEFT),  negative  horizontal  (DON'T  TURN  RIGHT, 
DON'T  TURN  LEFT),  positive  vertical  (CLIMB,  DESCEND),  negative  vertical 
(DON'T  CLIMB,  DON'T  DESCEND),  and  limited  vertical  (LIMIT  CLIMB  500  FPM, 
LIMIT  CLIMB  1000  FPM,  LIMIT  CLIMB  2000  FPM,  LIMIT  DESCENT  500  FPM,  LIMIT 
DESCENT  1000  FPM,  LIMIT  DESCENT  2000  FPM)  .  Commands  were  produced  on  the 
basis  of  an  ASA  detection  and  resolution  logic.  The  detection  logic  used 
the  same  type  of  criteria  as  the  PWI  logic  but  provided  protection  for  a 
smaller  volume  of  airspace.  The  resolution  logic  determined  the  proper 
command  to  resolve  the  given  conflict  situation.  Both  logics  were  executed 
once  per  scan  (4  seconds  for  full  mode  and  1  second  for  active  mode) .  A 
command  was  displayed  for  the  entire  time  the  protected  volume  was  violated. 
The  display  could  change  if  the  conflict  situation  became  more  or  less 
serious.  Positive  and  negative  commands  were  displayed  for  a  minimum  of 
5  seconds.  Limit  vertical  commands  were  displayed  for  a  miminum  of  1  second 

Figure  4-15  presents  the  results  of  questions  concerning  the  utility 
of  ASA  commands.  Most  pilots  felt  that  the  ASA  commands  were  usually,  if 
not  always,  clear  and  unambiguous  in  presenting  actions  for  the  pilot  to 
take  and  were  given  in  sufficient  time  to  avoid  a  potential  collision. 

The  pilots  did  not  always  agree  with  the  command  presented  but  were  satis¬ 
fied  with  the  command  most  of  the  time. 

The  question  concerning  accuracy  of  the  determined  commands  provides 
some  interesting  results.  Since  the  commands  were  produced  by  the  same 
logic  independently  of  the  display  used,  the  accuracy  of  the  commands  is 
uniform  over  all  three  displays.  However,  the  responses  are  distributed 
over  the  three  devices.  Since  there  is  no  difference  in  information  con¬ 
tent  of  the  commands,  presentation  of  the  commands  must  have  played  an 
important  role  in  the  decision  process.  The  responses  were  similar  to  the 
breakdown  of  pilots  by  overall  display  preference  (Subsection  4. 4. 1.6). 

Those  who  favored  the  IVSI  overall  felt  that  it  was  most  accurate  as  well. 
There  are  some  differences  in  the  LED  and  CRT  responses,  however.  The 
LED  display  scored  higher  in  accuracy  than  in  overall  preference,  with  the 
CRT  display  making  up  the  difference.  This  may  indicate  that  the  LED 
presentation  of  commands  (arrows  and  text)  may  be  the  preferred  presenta¬ 
tion  and  therefore  may  appear  more  accurate. 


Questions 

Answers 

In  general,  were  the 
actions  required  by  the 
commands  clear  and 

always 

unambiguous? 

usually 

IVSI  — 

LED  —  — 

seldom 

CRT  num 

never 

Do  you  feel  that  the 
commands  are  presented 

always 

in  sufficient  time  to 

avoid  a  potential 
collision? 

often 

IVSI  — 

LED  —  — 

sometimes 

CRT  . . 

never 

Did  you  agree 

always 

with  the  maneuver 

often 

command  given? 

seldom 

never 

Which  system  do  you 

IVSI 

feel  presented  the 
most  accurately 

LED 

determined  maneuver 
commands? 

CRT 

Percent  Responding 
25  50  75 


Questions 

Answers 

Are  all  of  the  command 
types  necessary? 

Positive  Commands 

Negative  Commands 

Limit  Commands 

yes 

no 

yes 

no 

yes 

no 

Active  BCAS  provides 
positive  commands,  negative 
commands,  and  limit 
commands.  Are  all  of 
these  necessary? 

Positive  Commands 

Negative  Commands 

Limit  Commands 

yes 

no 

yes 

no 

yes 

no 

When  presented  with 
negative  commands,  were 
you  able  to  continue  your 
intended  flight  path 
without  an  ATC  call? 

always 

often 

sometimes 

never 

When  presented  with 

always 

limit  commands,  were 

you  able  to  continue  your 

often 

original  flight  path 

sometimes 

without  a  call  to  ATC? 

never 

Percent  Responding 
25  50  75 
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Two  questions  on  the  necessity  of  the  three  command  types  (positive, 
negative,  and  limit  commands)  were  presented  —  one  in  the  debriefing 
questionnaire  and  one  in  the  supplemental  questionnaire.  The  purpose  of 
the  second  question  was  to  determine  if  the  pilots  had  changed  opinions, 
given  time  to  think  about  the  experiment.  The  original  responses  indicated 
a  favorable  reaction  to  all  of  the  command  types.  However,  the  supplemental 
question  indicates  a  change  of  opinion  with  regard  to  limit  commands.  The 
favorable  response  dropped  from  86  percent  of  the  respondents  (36  pilots) 
to  62  percent  of  the  respondents  (26  pilots)  for  the  same  sample  of  42 
responding  pilots.  Questions  on  the  other  two  command  types  (positive  and 
negative)  received  the  same  responses  as  the  original  question. 

The  last  two  questions  concerned  negative  and  limit  commands  and  tried 
to  determine  if  the  subject  pilots  were  able  to  continue  their  original 
flight  path  without  a  call  to  ATC.  The  consensus  for  both  command  types 
is  that  most  pilots  could  manage  without  ATC  calls  in  some  circumstances, 
and  almost  half  the  pilots  could  often  avoid  the  call. 

4.4.3  Impact  of  Introducing  Aircraft  Separation  Assurance  on  Flight  and 
ATC  Procedures 

This  section  examines  the  issues  of  pilot  workload,  system-generated 
maneuvers,  false  alarms,  pilot  confidence  in  the  system,  ASA  usefulness, 
and  impact  on  ATC. 

The  introduction  of  any  new  system  into  the  flight  deck  has  some 
impact  on  crew  workload.  Pilots  were  queried  specifically  on  this  subject; 
the  results  appear  in  Figure  4-16.  Most  pilots  felt  that  at  worst  the 
introduction  of  ASA  would  be  an  acceptable  increase  in  workload  and  that 
of  the  three  displays,  the  IVSI  would  be  least  unacceptable. 

Communications  probably  was  a  factor  in  those  results.  Figure  4-17 
shows  that  more  than  half  of  the  pilots  surveyed  felt  that  ASA  would 
require  more  communication  with  ATC.  The  greatest  concern  among  pilots 
is  causing  secondary  collisions  when  deviating  from  an  ATC  clearance  as 
a  result  of  an  ASA  command.  Therefore,  pilots  feel  that  any  deviations 
from  their  assigned  clearance  must  be  reported  to  ATC.  Furthermore, 
many  pilots  expect  that  they  will  contact  ATC  regarding  traffic  advisories 
of  aircraft  that  are  close  but  not  visually  acquired.  On  the  other  hand, 
there  are  some  pilots  who  feel  that  ATC  will  need  to  give  fewer  traffic 
advisories,  which  will  reduce  the  communications  load.  Nevertheless,  most 
pilots  felt  that  the  ATC  controller  should  be  kept  aware  of  any  commands 
given  by  ASA  in  the  form  of  either  voice  or  data  link  communication. 

The  effect  of  ASA-generated  command  maneuvers  is  investigated  in 
Figure  4-18.  The  results  show  that  there  was  seldom  a  strong  preference 
for  vertical  commands,  but  there  was  a  preference  for  horizontal  commands 
during  takeoff,  approach,  and  landing.  Those  pilots  who  preferred  hori¬ 
zontal  maneuvers  offered  the  following  justifications: 

There  is  less  probability  of  a  secondary  encounter  following  a 
horizontal  maneuver. 
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Figure  4-16.  PILOT  WORKLOAD 
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Figure  4-17.  COMMUNICATIONS  WORKLOAD 


•  The  idea  of  not  "busting"  altitude  is  ingrained  in  each  pilot's 
thinking  from  the  start  of  his  training. 

•  A  positive  G  load  provides  for  passenger  comfort,  passenger 
safety  (meal  service,  passengers  in  aisles,  etc.),  and  less 
passenger  anxiety  (passengers  are  often  unaware  of  horizontal 
maneuvers) . 

•  There  is  more  concern  about  terrain  problems  with  a  vertical  as 
opposed  to  a  horizontal  maneuver. 

•  With  gear  and  flaps  down,  many  aircraft  cannot  climb. 

•  No  power  changes  are  necessary. 

The  pilots  who  preferred  vertical  maneuvers  had  these  comments: 

•  Vertical  maneuvers  provide  a  more  rapid  flight-path  change  (at 
most  speeds)  than  horizontal. 

•  Vertical  commands  are  easier  to  respond  to. 

•  There  is  generally  more  vertical  airspace  available  for 
maneuvering. 

Regardless  of  the  maneuver  type,  there  was  considerable  concern  among 
the  pilots  about  maneuvering  into  another  aircraft.  This  is  borne  out  in 
specific  results  that  show  a  majority  at  least  sometimes  concerned.  While 
there  was  a  substantial  difference  between  the  displays,  the  IVSI  caused 
the  most  concern,  and  the  CRT  caused  the  least. 
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Figure  4-18.  SYSTEM-GENERATED  MANEUVERS 
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Magnitude  of  the  pilot  response  to  the  ASA  commands  is  examined  in 
Figure  4-19.  It  should  be  noted  that  these  results  reflect  the  pilots' 
own  perception  of  their  maneuvers .  Pilots  sometimes  perceived  larger  than 
normal  vertical  maneuvers  but  seldom  perceived  larger  than  normal  horizontal 
maneuvers . 


Figure  4-19.  MAGNITUDE  OF  PILOT  RESPONSE 


Confidence  issues  are  investigated  in  Figure  4-20.  Pilots  expressed 
increased  confidence  under  ASA  when  overflying  or  underflying  aircraft  by 
1,000  feet,  with  a  higher  degree  of  confidence  associated  with  the  LED  and 
CRT  displays.  However,  this  increased  confidence  was  not  enough  to 
encourage  reduced  lateral  separation.  Again,  more  confidence  was  placed 
in  the  LED  and  CRT  displays. 

Another  confidence  issue  concerns  unnecessary  alarms  —  those  alarms 
caused  by  situations  that  would  normally  be  resolved  by  the  planned  action 
of  either  or  both  aircraft.  This  type  of  situation  was  presented  on 
occasion  during  the  simulation  and  was  observed  by  the  pilots  as  illustrated 
in  Figure  4-21.  Pilots'  comments  on  this  situation  indicate  that  if  the 
pilot  knows  why  the  system  alarmed  because  of  available  supporting  informa¬ 
tion  (such  as  intruder  position  information) ,  then  these  "unnecessary" 
alerts  will  reinforce  rather  than  destroy  confidence  in  the  system.  However, 
the  lack  of  supporting  information  will  be  detrimental  to  crew  acceptance. 

It  was  also  stated  that  ASA  must  work  every  time  because  pilots  and  controllers 
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Figure  4-20.  PILOT  CONFIDENCE  IN  ASA 


would  tend  to  rely  on  the  system  very  heavily.  On  the  other  hand,  if 
the  system  proves  unreliable,  pilots  will  mentally  "tune  out"  the  display 
and  ignore  the  ASA  commands. 

When  asked  about  the  changes  that  would  be  required  in  aircraft 
operating  procedures  if  ASA  were  implemented,  most  pilots  felt  that  none 
were  required.  Those  who  commented  saw  added  duties  for  the  nonflying 
pilots  (monitoring  of  the  ASA  display) ,  a  requirement  to  equip  all  aircraft 
with  transponders,  and  changes  in  the  nominal  instrument  scan. 

Comments  on  changes  to  ATC  operations,  however,  included  the  entire 
spectrum  of  changes  —  from  no  changes  to  revamping  terminal  ATC.  Typical 
pilot  comments  are  as  follows: 

•  A  provision  required  for  immediately  alerting  ATC  of  a  deviation 
from  a  cleared  flight  path 

•  Fewer  aircraft  under  each  sector  controller 

•  Fewer  required  traffic  advisories 

•  More  traffic  advisories  required  to  prevent  unnecessary  maneuvering 
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Figure  4-21.  UNNECESSARY  ALARMS 


•  Increased  separation  in  terminal  areas  due  to  unanticipated 
aircraft  deviations 

•  Reduced  separation  in  some  areas 

•  Removal  of  the  250-knot  speed  limit  below  10,000  feet  —  increased 
traffic  flow 

•  No  changes  required 

More  than  80  percent  of  the  pilots  answered  the  question,  "Do  you  feel 
that  the  ASA  system  as  used  in  this  simulation  will  result  in  safer  opera¬ 
tion  in  respect  to  midair  collision?"  with  an  unqualified  yes,  and  comments 
such  as  "We've  needed  it  for  years"  and  "Yesterday  wasn’t  soon  enough"  were 
offered;  however,  most  pilots  felt  that  additional  testing  would  be  required 
before  implementing  ASA, 
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Of  the  38  pilots  who  expressed  opinions  on  additional  testing,  33 
pilots  felt  that  flight  tests  were  required,  2  pilots  suggested  additional 
simulator  tests,  and  only  3  pilots  felt  that  no  additional  testing  was 
required.  Those  who  suggested  flight  tests  were  divided  as  to  specific 
flight  test  objectives  and  parameters.  Some  recommended  routine  line 
operations  (11  replies) ,  some  suggested  stress  situations  (high-density 
airspace,  pilot  fatigue,  extreme  aircraft  limits  -  4  replies),  and  the 
rest  were  not  specific. 
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CHAPTER  FIVE 


QUANTITATIVE  DATA  ANALYSIS 


The  quantitative  data  analysis  consists  of  an  evaluation  of  pilot 
performance  and  command  effectiveness.  Pilot  performance  was  measured  in 
terms  of  pilot  response  (time  and  magnitude)  to  displayed  collision 
avoidance  commands,  and  command  effectiveness  was  evaluated  by  examining 
resultant  miss  distances. 

Since  pilot  response  time  is  one  of  the  key  factors  in  determining 
look-ahead  times  for  the  collision  avoidance  detection  logic,  the  analysis 
will  give  some  indication  of  the  adequacy  of  the  assumed  average  response 
time.  Response  time  can  also  be  used  to  provide  some  information  on 
display  efficiency.  If  it  can  be  assumed  that  crews  did  not  respond  until 
they  comprehended  the  conflict  situation  and  interpreted  the  displayed 
information,  significantly  different  response  times  for  the  three  displays 
could  reflect  the  efficiency  of  those  displays  or  the  information  presented 
on  them . 

Magnitude  and  rate  of  response  were  also  measured  as  indicators  of 
pilot  performance.  Pilots  were  briefed  before  the  simulation  sessions 
that  sufficient  warning  would  be  presented  by  the  collision  avoidance 
system  to  escape  a  conflict  situation  by  performing  standard-rate  hori¬ 
zontal  and  vertical  maneuvers .  Deviations  from  the  norm  could  indicate 
workload  problems  or  apprehension  with  the  conflict  situation. 

To  determine  command  effectiveness,  individual  conflicts  were  examined 
to  assess  the  cause  of  small  miss  distances  and  to  permit  developing 
recommendations  regarding  algorithmic  or  procedural  changes. 


5.1  DATA  COLLECTION 

The  quantitative  data  collected  during  this  evaluation  consist  of  a 
time-stamped  record  of  events  for  each  flight  scenario.  Data  were  recorded 
on  cockpit  simulator  parameters  (position,  velocity,  and  attitude),  command 
and  intruder  parameters,  and  simulation  controller  interaction. 

5.1.1  Simulator  Parameters 

•  •  • 

The  simulator  data  consist  of  position  (x,  y,  z) ,  velocity  (x,  y,  z) , 
and  attitude  data  (pitch  angular  rate,  roll  angle,  and  yaw  angle).  Pitch 
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angular  rate  and  roll  angle  data  were  updated  each  1/20  of  a  second.  All 
other  data  were  collected  at  1-second  intervals.  Position  and  velocity 
data  are  referenced  to  a  runway-oriented  coordinate  system  centered  at 
the  takeoff  point  of  the  Los  Angeles  International  Airport  runway  24R, 
with  the  Y-axis  along  the  runway  centerline  (magnetic  heading  248°)  and 
the  X-axis  offset  90°  in  a  clockwise  direction.  Twenty  successive  pitch 
angular  rate  and  roll  angle  data  samples  were  accumulated  in  arrays  and 
recorded,  together  with  the  remainder  of  the  simulator  data,  at  1-second 
intervals.  Each  record  was  time-stamped,  with  the  time  being  expressed  in 
seconds  following  the  initiation  of  simulation. 

5.1.2  Command  and  Intruder  Data 


At  each  occurrence  of  a  new  command,  the  command  (coded  in  a  simple 
format)  was  recorded,  together  with  an  index  into  the  track  file  of  the 
aircraft  causing  the  command  and  the  time  at  which  the  new  command  was 
initiated  relative  to  simulation  initiation.  Miss  distance  information 
was  updated  continuously  and  written  to  dis.k  when  the  display  was  cleared. 
The  record  included  the  point  of  closest  approach  (range  and  altitude) ,  the 
time  at  which  the  point  was  reached,  and  the  time  at  which  the  display  was 
cleared. 

While  it  was  a  simulation  controller  input  that  released  an  intruder 
aircraft  into  the  simulation,  the  track  was  not  initiated  until  the  simula¬ 
tor  was  a  fixed  distance  from  a  fixed  reference  point.  At  that  time,  a 
time-stamped  record  was  written  to  disk;  it  included  track  ID,  initial 
position,  initial  heading,  and  initial  speed. 

5.1.3  Simulation  Controller  Interaction 


The  simulation  controller  had  the  ability  to  request  information  about 
simulation  activities  and  make  modifications  to  the  simulation  to  control 
those  activities.  Since  these  modifications  were  unique  to  each  simula¬ 
tion  session,  a  time-stamped  record  was  written  to  disk  at  each  occurrence. 
The  simulation  controller  inputs  are  as  follows: 

1.  Change  Mode.  ASA  mode  changed  from  full  to  active  and  back  again 

2.  Kill  Aircraft.  Eliminate  an  aircraft  from  the  simulation  (track 
ID  dumped) 

3.  Altitude-Speed  Change.  Modify  the  flight  plan  of  the  intruder 
aircraft  to  ensure  a  collision  (track  ID  and  magnitude  of 
altitude/speed  change) 


5.2  RESPONSE-TIME  ANALYSIS 

Pilot  response  time  is  defined  as  the  time  elapsed  from  command  pre¬ 
sentation  to  maneuver  initiation.  Maneuver  initiation  is  defined  as  a 
1-degree  change  in  roll  angle  (horizontal  maneuvers)  or  a  1/2-degree-per- 
second  change  in  pitch  rate  (vertical  maneuvers) .  These  values  were 
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selected  empirically  by  examining  the  data  and  observing  how  maneuvers 
evolved  from  steady-state  conditions. 

The  response  times  were  generated  for  each  command  and  were  labeled 
according  to  its  associated  display  type  (IVSI,  LED,  or  CRT) ,  ASA  mode 
(active  or  full) ,  scenario  (1  to  6) ,  and  command  type  (CLIMB,  DESCEND, 

DON'T  CLIMB,  DON'T  DESCEND,  LIMIT  CLIMB  500,  1,000,  or  2,000  feet  per 
minute,  LIMIT  DESCENT  500,  100,  or  2,000  feet  per  minute,  TURN  RIGHT,  TURN 
LEFT,  DON'T  TURN  RIGHT,  AND  DON'T  TURN  LEFT).  The  response  times  were 
recorded  to  1-second  accuracy.  However,  the  averages  are  presented  to 
three  decimal  digits  to  permit  better  comparisons  between  them.  Since 
less  than  1  percent  of  all  response  times  exceeded  10  seconds  (6  out  of 
656  response  times) ,  all  such  "outliers"  were  eliminated  from  the  data 
sets.  In  addition,  five  of  six  large  response  times  were  associated  with 
negative  or  limit  commands.  Since  these  commands  may  not  have  required 
any  action  from  the  pilot,  the  large  response  times  may  have  been  a  result 
of  normal  flight  path  changes. 

Frequency  distributions  of  response  times  were  created  for  each  of 
the  factors  of  interest  in  the  evaluation  —  display  type,  ASA  mode, 
scenario,  and  command  type.  An  examination  of  response  times  disclosed 
that  the  data  were  not  normally  distributed.  A  gamma  distribution  yielded 
a  goodness-of-f it  of  0.922,  which  was  not  considered  especially  conclusive. 
Curve-fitting  with  other  common  distributions  was  similiarly  inconclusive; 
distribution-free  (nonparametric)  methods  were  therefore  used  to  correlate 
response  times  to  the  various  influencing  factors. 

To  simplify  comparisons  of  the  distributions,  the  frequency  data  were 
converted  into  probability  of  occurrence,  which  is  defined  as  the  number 
of  occurrences  for  each  time  interval  divided  by  the  number  of  occurrences 
for  all  time  intervals.  The  Kolmogorov-Smirnov  test  was  used  as  a  com¬ 
parison  test  between  distributions.  The  test  examines  the  difference 
between  two  distributions  and  determines  whether  the  differences  are  due 
to  chance  or  the  distributions  are  significantly  different.  For  multiple 
comparisons  involving  several  distributions,  a  rank  test  analogous  to  the 
classical  Analysis  of  Variance  (ANOVA)  method  was  employed.  This  permits 
the  simultaneous  study  of  the  effects  of  several  factors.  The  Kruskal- 
Wallis  One-Way  Analysis  of  Variance  was  used  to  test  the  null  hypothesis 
that  there  was  no  significant  difference  among  the  expected  values  of 
several  populations.  Observed  differences  in  samples  were  tested  to 
determine  whether  they  were  true  differences  or  random  variations  resulting 
from  "noise"  in  the  data.  The  distribution-free  method  applied  uses 
numbers  that  correspond  to  the  relative  ranking  of  data  rather  than  the 
data  themselves.  A  more  detailed  description  of  these  tests  is  presented 
in  Appendix  F. 

For  each  statistical  test,  an  initial  hypothesis  about  the  nature  of 
the  responses  is  proposed,  together  with  an  alternate  hypothesis.  The 
objective  of  the  test  is  to  determine  whether  to  accept  or  reject  the 
original  hypothesis  in  favor  of  the  alternate  hypothesis.  The  level  of 
significance  is  the  probability  of  rejecting  the  original  hypothesis  in 
favor  of  the  alternate  when  the  original  hypothesis  is  actually  true.  The 
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tests  in  this  analysis  were  conducted  at  the  five  percent  level  of  signifi¬ 
cance  to  minimize  the  chance  of  drawing  an  incorrect  conclusion  from  the 
data. 


The  probability  distributions  for  measured  response  times  appear  in 
Figures  5-1  through  5-5.  Figure  5-1  presents  the  probability  distribution 
for  the  total  population  of  measured  response  times.  The  mean  for  overall 
response  time  is  3.185  seconds,  with  a  standard  deviation  of  1.823  seconds. 
The  response  times  observed  in  this  simulation  appear  to  be  consistent 
with  previous  studies  that  involved  air  transport  aircraft.*  These  studies 
assumed  a  1.5-second  pilot  response  and  1-second  servo  delay,  or  2.5- 
second  total  response  time.  The  observed  response  times  are  significantly 
less  than  the  6-second  values  observed  in  some  NAFEC  studies.**  However, 
there  were  some  differences  in  the  experimental  approach.  The  NAFEC 
simulation  used  one-man  general  aviation  crews  in  a  GAT  II  trainer.  This 
study  used  two-  and  three-man  crews  of  professional  airline  pilots  in  a 
B-727  simulator.  Further,  in  the  NAFEC  test  an  audible  alarm  sounded  for 
positive  commands  only,  while  in  this  test  series  it  sounded  for  all 
commands . 

Since  many  of  the  conflict  situations  resulted  in  more  than  one  com¬ 
mand,  the  impact  of  subsequent  commands  on  response  time  was  also  investi¬ 
gated.  Figure  5-1  shows  the  response  times  for  the  first  command  in  a 
command  sequence.  The  resultant  mean  is  3.274  seconds,  with  a  standard 
deviation  of  1.652  seconds.  The  increase,  when  compared  with  the  total 
population's  average  response  times,  is  probably  due  to  the  pilot's 
awareness  of  a  conflict  situation  at  the  time  of  the  subsequent  commands 
and  the  additional  attention  given  to  the  collision  avoidance  display. 

The  data  were  broken  down  by  display  type  (IVSI,  LED,  and  CRT)  and 
analyzed  by  using  the  Kurskal-Wallis  methods  for  multiple  comparisons. 
Figure  5-2  illustrates  this  breakdown  by  display  type  for  the  total  popula¬ 
tion  of  response  times.  The  results  show  a  statistically  significant 
difference,  with  the  IVSI  demonstrating  the  shortest  response  time  (2.996 
seconds);  the  LED  was  second  (3.123  seconds),  and  the  CRT  produced  the 
longest  overall  response  time  (3.468  seconds).  However,  these  differences 
do  not  appear  to  have  any  operational  significance.  The  differences  could 
reflect  the  complexity  of  the  display,  its  location  with  respect  to  the 
normal  instrument  scan,  or  some  combination  of  the  two  factors.  The  IVSI 
was  the  simplest  display  to  understand  and  was  located  in  the  normal 
instrument  scan.  The  CRT  was  the  most  complex  display  to  understand  and 
was  located  outside  the  normal  instrument  scan. 


*Evaluation  of  Threat  Logic  for  the  ATA  ACAS ,  Watson,  Lazzareschi,  Wedlake, 
Report  No.  MDC  E0116,  13  March  1970. 

"Analysis  of  Warning  Times  for  Collision  Avoidance  Systems",  John  M.  Holt, 
Ronald  M.  Anderson,  IEEE  Transactions  on  Aerospace  and  Electronic  Systems, 
March  1968. 

**Modeling  Pilot  Response  Delays  to  BCAS  Commands,  Report  No.  FAA-RD-79-74 , 
October  1979. 
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The  impact  of  subsequent  commands  in  a  command  sequence  by  display 
type  was  also  investigated.  The  probability  distributions  are  presented 
in  Figure  5-3,  and  the  entire  breakdown  of  response  time  by  display  type 
is  summarized  in  Table  5-1.  With  the  exception  of  the  IVSI,  initial  com¬ 
mands  produced  a  longer  response  time,  and  subsequent  commands  were,  on 
the  average,  1/2  second  shorter.  The  response  times  for  the  IVSI  were 
statistically  the  same,  indicating  again  that  the  positioning  of  the 
instrument  in  the  normal  scan  may  be  significant. 

The  Kruskal-Wallis  comparison  test  was  also  applied  to  response  time 
data  across  scenarios  for  the  first  command  in  a  sequence.  The  results 
of  the  test  showed  that,  as  expected,  there  are  no  statistically  signifi¬ 
cant  differences  between  response  times  over  the  six  scenarios.  The 
probability  distributions  of  these  response  times  appear  in  Figure  5-4. 

The  effect  of  ASA  mode  (full  versus  active)  on  response  times  was 
analyzed  by  using  the  Kolmogorov-Smirnov  comparison  test.  The  data  used 
were  those  of  the  first  command  in  a  sequence.  The  results  indicate  that 
there  are  significant  differences  in  response  between  the  two  modes  of 
operation,  with  full  mode  requiring,  on  the  average,  0.4  second  longer. 
Since  full  mode  includes  both  horizontal  and  vertical  commands  and  active 
mode  includes  vertical  commands  only,  the  effect  of  horizontal  versus 
vertical  commands  was  investigated.  A  second  comparison  was  performed 
between  full  mode  (vertical  commands  only)  and  active  mode  (all  commands) . 
The  results  show  no  significant  difference  in  response  times,  indicating 
that  horizontal  maneuvers  increase  pilot  response  time.  The  results  are 
summarized  in  Table  5-2,  and  the  frequency  distributions  appear  in  Figure 
5-5. 


The  last  influencing  factor  investigated  was  the  command  type. 

Again,  the  first  command  in  the  sequence  was  representative  of  the  actual 
response  time. 

The  first  test  performed  compared  pairs  of  commands  that  were  of  the 
same  type  but  opposite  in  direction  (e.g.,  CLIMB  versus  DESCEND,  TURN 
RIGHT  versus  TURN  LEFT,  DON'T  CLIMB  versus  DON'T  DESCEND,  etc.).  There 
were  no  significant  differences  in  any  of  the  pairs  tested. 

The  second  test  compared  the  four  pairs  of  positive  and  negative  com¬ 
mands  (the  sample  size  was  too  small  to  compare  limit  commands) .  Negative 
commands  were  considered  only  if  an  action  was  taken  by  the  pilot.  Analysis 
of  variance  indicated  that  horizontal  commands  require  more  time  than 
vertical  commands.  This  is  verified  by  visual  inspection  of  the  averages 
summarized  in  Table  5-3. 
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Figure  5-3.  PROBABILITY  DISTRIBUTIONS,  FIRST  COMMAND/SUBSEQUENT 
COMMAND  BY  DISPLAY  TYPE 
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Figure  5-4.  PROBABILITY  DISTRIBUTIONS  —  BY  SCENARIO  (RESPONSES 
TO  FIRST  COMMAND  IN  SEQUENCE) 
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Table  5-2.  MEAN  RESPONSE  TIME  SUMMARY  BY  MODE 

Response  Time  to  First  Command  j 

Mode 

in  a  Sequence 

Number 

Mean 

Standard 

Samples 

( Specifications ) 

Deviation 

Full  (All  Commands) 

239 

3.402 

1.849 

Full  (Vertical  Commands) 

144 

3.028 

1.613 

Active  (All  Commands) 

196 

3.092 

1.722 

5.3  MAGNITUDE  OF  RESPONSE 

Subject  pilots  were  advised  before  the  simulator  session  of  the 
following  collision  avoidance  procedures: 

•  Initiate  an  escape  maneuver  if 

••  A  conflict  is  perceived  in  the  cockpit  visual  system. 

••  A  conflict  is  perceived  on  the  basis  of  displayed  traffic 
advisory  information. 

• •  A  command  is  presented 

•  Respond  to  the  command  in  the  following  maneuver: 

••  Climb  Maneuver.  Rotate  to  a  pitch-up  attitude  approximating 
a  go-around  configuration. 

••  Descent  Maneuver.  Reduce  thrust.  Pitch  over  at  an  attitude 
approximating  a  profile  descent. 

••  Turn  Maneuver.  Roll  into  a  30°  bank  in  the  direction  of  the 
command . 

••  Limit  Vertical  Command.  Obey  the  limit  instruction. 

••  No  Turn  Command.  Cease  or  avoid  turning  in  the  direction  of 
the  command . 

•  Terminate  an  escape  maneuver  if 

••  The  displayed  command  is  cleared. 

••  The  conflict  no  longer  exists  and  the  display  is  clear. 

Pilots  were  specifically  instructed  to  perform  normal  maneuvers  at 
standard  rates  to  avoid  a  collision.  Therefore,  it  can  be  assumed  that 
rates  in  excess  of  nominal  values  indicate  an  apprehension  about  the 
system.  There  is  great  concern  among  airline  pilots  regarding  passenger 
comfort  and  safety,  and  if  maneuvers  with  excessive  rates  are  an  instinctive 
reaction  to  ASA,  the  system  may  ultimately  prove  to  be  unacceptable. 
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Table  5-3.  MEAN  RESPONSE  TIME  BY 

COMMAND 

Response  Time  to  First 

Command  in  a  Sequence 

Mode 

Standard 

Deviation 

DESCEND 

3.195 

1.703 

CLIMB 

2.915 

1.213 

DON'T  DESCEND 

2.917 

1.933 

DON'T  CLIMB 

2.975 

1.625 

DON'T  TURN  RIGHT 

4.391 

1.877 

DON'T  TURN  LEFT 

4.000 

1.414 

TURN  RIGHT 

3.808 

0.939 

TURN  LEFT 

3.950 

1.154 

Magnitude  of  pilot  response  was  measured  in  terms  of  roll  angle  and 
pitch  rate.  Maximum  values  achieved  were  determined  for  each  positive 
command  encountered.  A  frequency  distribution  of  maximum  roll  angles  is 
illustrated  in  Figure  5-6. 

There  were  significantly  more  left  turn  commands  than  right  turn  com¬ 
mands  because  of  the  nature  of  the  scenarios  and  conflict  geometries  used. 

A  comparison  test  showed  that  there  was  no  significant  difference  between 
the  distributions  of  right  and  left  turn  maximum  roll  angles.  The  results 
are  centered  on  a  30°  bank  for  both  right  and  left  turns,  as  expected  on 
the  basis  of  the  instructions  that  the  pilots  received  before  the  simulator 
session.  Some  of  the  larger  responses  can  be  explained  as  follows: 

*  Slow  response  to  first  command  in  sequence  (positive  command  was 
a  follow-up) 

*  Simulator  already  in  left  bank 

*  Apprehension  due  to  lack  of  visual  acquisition 

The  last  reason  was  the  one  given  most  often  by  the  pilots  for  large 
maneuver  rates.  The  rates  were  not  held  for  very  long,  however.  The  large 
maximum  roll  angles  were  generally  no  more  than  1  second  in  duration  before 
the  pilot  began  rolling  out  of  the  bank. 

Maximum  pitch  rates  were  also  analyzed.  Figures  5-7  and  5-8  show  the 
probability  distributions  for  descend  and  climb  commands  for  all  commands 
and  for  the  first  command  in  a  sequence.  Comparison  tests  performed  on 
the  data  detected  no  significant  differences  in  the  magnitude  of  vertical 
maneuvers  between  all  commands  and  first  command  in  a  sequence.  Differences 


5-13 


AD-A083  986  ARINC  RESEARCH  CORP  ANNAPOLIS  MD  F/G  1/2 

AN  EVALUATION  OF  AIRCRAFT  SEPARATION  ASSURANCE  CONCEPTS  USING  A— ETC(U) 
NOV  79  B  MORGENSTERN#  T  P  BERRY  DOT-FA78WA-4091 

UNCLASSIFIED  1343-01-3-2058-VOL-l  FAA/RD-79-124-1  NL 


Figure  5-6.  MAXIMUM  ROLL  ANGLE  -  POSITIVE  HORIZONTAL  COMMANDS 
TURN  RIGHT  AND  TURN  LEFT 
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Figure  5-7.  PROBABILITY  DISTRIBUTION,  DESCEND  COMMAND,  MAXIMUM 
PITCH  RATE 
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in  the  mean  and  standard  deviation  for  climb  and  descend  maneuvers  were 
investigated  by  using  the  classical  F-Test  and  t-Test,  respectively.  At 
the  0.05  level  of  significance,  there  was  a  difference  in  pitch  rate 
between  climb  and  descend  maneuvers ;  climb  commands  produced  higher  pitch 
rates  than  descend  commands . 

A  data  breakdown  by  display  and  for  all  command  types  appears  in 
Table  5-4.  Analysis  shows  that  there  are  significant  differences  in  mean 
maximum  pitch  rate  for  the  three  displays.  The  smallest  pitch  rates  were 
associated  consistently  with  the  CRT  display.  This  could  indicate  less 
apprehension  with  the  conflict  situation  given  the  CRT-type  presentation. 


5.4  MISS  DISTANCE  ANALYSIS 

This  section  examines  the  point  of  closest  approach  between  conflict 
pairs  that  included  the  cockpit  simulator.  As  described  in  Chapter  Three, 
conflict  situations  were  preprogrammed  to  occur  six  times  in  a  given 
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scenario.  This  number  varied  because  some  conflicts  did  not  materialize 
and  others  were  unplanned  (involving  background  traffic) .  The  conflicts 
were  designed  to  be  "zero-zero"  encounters;  however,  variations  in  the  way 
pilots  flew  the  simulator  provided  a  wide  range  of  miss  distances .  Some 
control  of  the  intruder  was  available  (Subsection  3.5.2),  but  it  was  used 
only  to  guarantee  a  displayed  command.  Once  a  command  was  generated,  all 
control  of  the  intruder  ceased.  This  permitted  the  pilot  to  evade  con¬ 
flicting  traffic  as  he  would  in  a  normal  line  operation. 

The  miss  distances  were  recorded  for  516  out  of  the  576  conflicts 
(miss  distance  was  not  recorded  for  the  first  three  crews) .  Table  5-5 
shows  the  breakdown  of  these  data  by  display  and  indicates  the  number  of 
close  encounters  that  are  defined  as  those  encounters  in  which  the  achieved 
miss  distance  was  less  than  1/2  mile  in  range  and  500  feet  in  altitude 
(3,079)  feet  slant-range).  The  results  indicate  that  significantly  more 
conflict  situations  developed  with  the  IVSI  than  with  the  LED  or  CRT.  This 
can  be  explained  by  the  absence  of  traffic  advisories  with  the  IVSI.  Pilots 
using  the  LED  and  CRT  displays  often  used  the  traffic  advisory  information 
to  anticipate  a  conflict,  which  generally  reduced  the  number  and  duration 
of  commands  and  in  some  cases  avoided  the  conflict  situation  entirely. 
Another  significant  result  was  the  reduction  in  close  encounters  when  the 
CRT  display  was  used,  indicating  that  the  pilots  were  able  to  use  the  dis¬ 
play  to  maximize  their  achieved  miss  distance. 

Figure  5-9  shows  the  distribution  of  the  achieved  miss  distances  for 
the  close  encounters .  Only  seven  (one  percent)  conflict  situations  pro¬ 
duced  miss  distances  less  than  500  feet  slant  range.  The  smallest  miss 
distance  was  355  feet  (300  feet  of  which  was  vertical  separation) .  The 
pilot  was  a  second  officer  with  5,000  hours  (150  hours  in  past  year),  and 
the  small  miss  distance  was  a  result  of  a  shallow  descent  maneuver  as  a 
response  to  a  DESCEND  command.  The  same  pilot  responded  to  a  CLIMB  command 
with  a  shallow  climb  that  resulted  in  another  achieved  miss  distance  less 
than  500  feet.  The  remaining  five  "near  midairs"  (Figure  5-9)  were  all  in 
the  range  of  450  to  500  feet  in  achieved  miss  distance.  The  achieved 
vertical  separations  ranged  from  160  to  430  feet. 

Causes  of  small  miss  distances  included  the  following: 

•  Slow  response  to  commands 

•  Inadequate  aircraft  maneuvering 

•  Unfamiliarity  with  the  United  B-727  simulator 

•  "Tail  chase"  (intruder  following  own  aircraft)  conflict  situations 

It  should  be  noted,  however,  that  small  miss  distances  (0.5  miles  in  range, 
500  feet  in  altitude)  occurred  even  when  the  pilots  responded  as  instructed. 
On  the  other  hand,  none  of  these  situations  resulted  in  an  unacceptable 
miss  distance. 

Another  important  result  illustrated  in  Figure  5-9  is  the  breakdown  of 
close  encounters  by  display.  No  achieved  miss  distances  of  less  than  500 
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Table  5-5.  DISTRIBUTION  OF  ACHIEVED 
MISS  DISTANCE 
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Figure  5-9.  DISTRIBUTION  OF  MISS  DISTANCES  -  CLOSE  ENCOUNTERS 
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feet  occurred  with  the  CRT,  and  only  three  occurred  between  500  and  1,000 
feet  (one  was  542  feet  as  a  result  of  a  "tail  chase,"  or  intruder-following- 
own-aircraft  geometry  as  depicted  in  Figure  5-10;  the  other  two  resulted 
in  approximately  900  feet  of  achieved  miss  distance).  Overall,  pilots 
flying  with  the  CRT  (accounting  for  33  percent  of  all  flights)  produced 
only  20  percent  of  the  close  encounters. 


Figure  5-10.  TAIL-CHASE  GEOMETRY 


the  two  or  three  most  important,  with  preference  shown  for  a  pictorial 
rather  than  an  alphanumeric  presentation. 

Pilots  felt  that  ASA  commands  were  presented  in  a  clear  and  unambiguous 
fashion  and  in  sufficient  time  to  avoid  a  potential  collision.  While  the 
suggested  maneuvers  were  not  always  what  the  pilots  had  in  mind,  they  were 
generally  satisfied  with  the  results.  Positive,  negative,  and  limit  com¬ 
mands  are  all  considered  useful  commands,  but  there  is  less  agreement  on 
limit  commands. 

6.1.2  Display  Preference 

There  is  a  definite  preference  for  the  CRT-type  display  presentation 
(alphanumeric  commands  with  a  pictorial  traffic  situation) .  The  CRT  was 
ranked  first  by  almost  50  percent  of  the  pilots  surveyed.  The  LED  display 
was  ranked  second  and  the  IVSI  third.  A  significant  result,  however,  is 
that  a  large  percentage  (73  percent)  of  pilots  surveyed  would  find  any  of 
the  three  displays  acceptable.  Analysis  of  these  results  by  scenario, 
pilot  experience  (captains  versus  first  officers) ,  and  recommending  organi¬ 
zation  (ALPA  versus  airline  management)  indicated  similar  findings. 

Combinations  of  displays  were  investigated;  more  than  50  percent  of 
the  pilots  favored  combining  the  ASA  function  with  some  other  display.  The 
IVSI  and  CRT  combination  was  best  received,  with  26  percent  of  the  pilots 
in  agreement,  and  the  IVSI  and  LED  combination  and  LED  and  CRT  combination 
each  received  15  percent  of  the  pilot's  votes. 

Alternate  aircraft  instruments  that  could  be  modified  to  provide  ASA 
information  were  recommended  by  a  third  of  the  pilots  surveyed.  The 
instruments  most  often  recommended  were  the  horizontal  situation  indicator, 
the  artificial  horizon/flight  director,  and  the  weather  radar. 

6.1.3  Display  Presentation 

This  section  examines  the  test  display  evaluation  and  suggests  those 
features  which  are  most  favored  for  a  generic  ASA  display. 

Location  is  a  key  consideration.  Many  of  the  negative  comments  on  the 
LED  and  CRT  displays  related  to  their  location  on  the  instrument  panel  with 
respect  to  a  normal  instrument  scan.  Location  of  the  IVSI  probably  had  a 
significant  effect  on  its  acceptance. 

Color  is  a  very  desirable  feature  on  an  ASA  display  for  improved  inter¬ 
pretation  of  the  presented  information,  and  the  use  of  red  as  an  indication 
of  a  positive  action  is  acceptable. 

An  aural  alerting  feature  is  considered  a  necessary  evil.  Most  pilots 
felt  that  an  aural  alert  was  required;  however,  there  were  strong  warnings 
against  a  "startling  alarm." 


6-2 


Character  size  and  symbolic  presentation  are  also  important  factors. 
Pilots  want  easy  readability  of  alphanumeric  text  and  consider  arrows  to 
provide  the  best  presentation  of  positive  commands.  In  addition,  too  many 
alphanumerics  reduce  readability;  unnecessary  text  should  be  eliminated. 

6.1.4  System  Implementation 

This  section  examines  the  issues  of  ASA  maneuvers,  system  effective¬ 
ness,  the  impact  of  ASA  introduction  on  ATC  and  flight  procedures,  and 
additional  testing. 

The  introduction  of  ASA  to  the  flight  deck  has  added  what  pilots 
consider  an  acceptable  increase  in  workload  for  the  nonflying  pilot.  More 
voice  communication  is  expected  unless  the  ASA  commands  can  be  electron¬ 
ically  transmitted  to  the  air  traffic  controller.  Additional  voice  com¬ 
munication  may  also  occur,  initially,  to  verify  traffic  that  appears  on  the 
ASA  display  but  is  not  visually  acquired. 

Pilots  are  concerned  about  maneuvering  into  other  aircraft  as  a  result 
of  an  ASA  command.  This  is  one  of  the  main  reasons  for  interest  in  dis¬ 
played  traffic  advisories,  and  a  reason  given  for  preference  of  horizontal 
commands  over  vertical.  This  preference  is  not  shared  by  all  pilots,  how¬ 
ever.  While  horizontal  maneuvers  are  generally  preferred  for  takeoff, 
approach,  or  landing  phases,  there  was  no  strong  preference  for  either 
horizontal  or  vertical  maneuvers  during  other  phases  of  flight.  The  pilots 
who  preferred  horizontal  maneuvers  cited  passenger  comfort  and  safety  as  a 
key  consideration  in  maneuvering  the  aircraft.  Horizontal  maneuvers  pro¬ 
vide  a  guaranteed  positive  G  load  and  less  passenger  anxiety,  seldom  require 
power  changes,  are  generally  unaffected  by  gear  and  flap  configurations,  and 
are  independent  of  terrain  problems.  Pilots  favoring  vertical  maneuvers 
cited  more  maneuvering  room,  easier  response,  and  more  rapid  flight  path 
change  (at  most  speeds) . 

Response  to  commands  was  somewhat  faster  for  vertical  versus  horizontal 
maneuvers,  but  the  difference  was  only  1  second,  which  does  not  have  a  major 
impact  on  the  system  design.  Average  response  time  ranged  from  3  to  3.5 
seconds.  The  IVSI,  which  was  the  least  complex  and  most  "in  scan"  instru¬ 
ment,  provided  the  quickest  response;  and  the  CRT,  which  was  the  most  com¬ 
plex  and  farthest  "out  of  scan"  instrument,  provided  the  slowest  response. 
The  response  time  was  defined  as  the  time  from  command  presentation  to 
maneuver  initiation  and  was  measured  to  1 -second  accuracy.  Response  to 
commands  was  not  excessive  in  either  the  vertical  or  horizontal  dimension 
and  was  not  perceived  by  the  pilots  as  excessive. 

Pilots  expressed  increased  confidence  while  flying  with  ASA  but  not 
enough  to  permit  reducing  lateral  separation  standards.  Concern  was 
expressed  about  unnecessary  alarms,  but  this  was  not  considered  a  problem 
if  pilots  were  provided  supporting  information  as  to  why  the  alarm  went 
off.  Pilots  were  concerned  about  system  reliability,  however.  They  felt 
that  the  system  must  work  all  of  the  time  or  it  would  be  ignored  by  the 
pilot. 
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The  system  was  effective  in  producing  a  500-foot  slant  range  miss  dis¬ 
tance  in  99  percent  of  the  conflict  situations.  In  some  of  these  "close" 
situations,  the  pilots  were  unfamiliar  with  the  aircraft  or  less  experienced. 
The  resolution  logic  was  least  effective  in  a  "tail  chase"  geometry,  where 
an  intruder  aircraft  was  descending  on  top  of  the  simulator  and  only  verti¬ 
cal  maneuvers  were  available  for  resolution  (active  mode) .  The  number  of 
"close"  situations  decreased  inversely  to  the  amount  of  supporting  informa¬ 
tion  on  the  display,  and  none  of  these  situations  occurred  while  the  CRT 
display  was  in  use.  In  fact,  the  total  number  of  conflicts  was  significantly 
higher  with  the  IVSI  because  of  a  lack  of  supporting  information  (traffic 
advisories) . 

Other  criticism  of  the  resolution  logic  reflected  dissatisfaction  in 
the  updating  of  limit  vertical  commands .  The  current  logic  allows  vertical 
limit  commands  to  be  changed  as  often  as  once  per  second  in  active  mode. 

This  rate  is  perceived  as  too  rapid,  and  it  causes  confusion.  Positive  and 
negative  commands  had  to  be  displayed  for  5  seconds  before  being  updated. 

The  pilots  were  more  comfortable  with  the  5-second  rate. 

Pilots  felt  that  few  or  no  changes  would  be  required  in  flight  pro¬ 
cedures.  Those  who  suggested  changes  saw  added  duties  for  the  nonflying 
pilot  (monitoring  the  ASA  display) ,  a  requirement  to  equip  aircraft  with 
transponders,  and,  potentially,  changes  to  the  nominal  instrument  scan. 

Changes  to  ATC  procedures  centered  primarily  on  the  problem  of  unex¬ 
pected  deviations  caused  by  ASA  maneuvers.  Pilots  foresee  the  requirement 
of  immediate  notification  of  flight  path  deviations  to  the  air  traffic 
controller.  Some  traffic  advisories  may  have  to  be  issued  by  ATC  to  prevent 
unnecessary  maneuvering,  while  others  might  be  eliminated.  Separation 
standards  might  have  to  be  increased  in  terminal  areas  because  of  unantici¬ 
pated  deviations  and  may  be  reduced  in  other  areas . 

While  pilots  wanted  the  system  "yesterday,"  most  believed  that  addi¬ 
tional  testing  was  required.  They  suggested  flight  tests,  and  those  who 
specified  the  type  of  testing  recommended  line  operation  testing. 


6.2  RECOMMENDATIONS 
6.2.1  Flight  Test 

It  is  recommended  that  consideration  be  given  to  both  operational  and 
experimental  flight  tests.  The  ASA  logic  should  be  subjected  to  flight 
testing  in  an  actual  airline  operational  environment  to  achieve  the  follow¬ 
ing  objectives: 

•  Determine  the  incidence  of  ASA  alarms 

•  Determine  the  incidence  of  false  ASA  a) arms 

•  Establish  the  sensitivity  of  ASA  performance  to  aircraft  density 
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Identify  pilot  reactions  to  ASA  commands  in  actual  flight  conditions 


Experimental  flight  tests  should  be  conducted  by  the  FAA  to  stress  the  ASA 
logic  and  determine  the  satisfactory  operation  and  reliability  of  the  experi¬ 
mental  ASA  equipment . 

6.2.2  ASA  Logic  Modifications 

It  is  recommended  that  the  ASA  logic  be  modified  to  provide  the  same 
minimum  display  time  for  limit  commands  as  provided  for  positive  and  nega¬ 
tive  commands.  In  addition,  specific  geometries,  such  as  the  "tail  chase" 
geometry,  should  be  further  investigated  to  determine  if  the  situation  might 
be  improved  by  procedural  instructions. 

6.2.3  ASA  Displays  I 

The  design  and  selection  of  the  best  ASA  display  is  dependent  on  the 
aircraft's  normal  operational  environment  and  should  be  determined  by  the 
aircraft  operator.  Specific  consideration  should  be  given  to  display  loca¬ 
tion,  use  of  color,  inclusion  of  traffic  advisories,  symbolic  representation 
of  information,  and  combined  function  displays  (normal  instrument  function 
plus  ASA) . 

6.2.4  ATC  and  Flight  Procedures 

Before  implementation  of  an  ASA  system,  a  set  of  operational  procedures 
should  be  developed  in  the  areas  of  response  to  ASA  traffic  advisories, 
response  to  ASA  commands,  and  communication  with  ATC.  during  a  conflict  situ¬ 
ation.  A  training  program  should  be  established  to  explain  how  the  system 
functions,  what  each  type  of  command  advisory  represents  in  terms  of  a  traf¬ 
fic  situation,  and  the  command  sequences  that  might  be  produced.  Normally, 
the  development  of  pilot  training  programs  and  flight  procedures  is  the 
responsibility  of  the  aircraft  operator,  subject  to  the  approval  of  the 
FAA.  It  is  expected  that  pilot  procedures  will  be  developed  by  a  committee 
composed  of  airline  industry  personnel  and  approved  by  the  Federal  Aviation 
Administration. 
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